Reproduce “Read Between the Layers: Leveraging Multi-Layer Representations for Rehearsal-Free Continual Learning with Pre-Trained Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kyra Ahrens, Hans Hergen Lehmann, Jae Hee Lee, Stefan Wermter
LayUP is a rehearsal-free continual learning method for pre-trained models that leverages multi-layer feature representations and second-order statistics to construct robust class prototypes. This approach demonstrates improved performance across various incremental learning settings, especially when faced with large domain shifts, and achieves 75-90% reductions in memory and runtime compared to several baselines.
Key insights
- Intermediate layers of pre-trained models contain class-discriminative information that is often more robust to domain shifts than features from the final layer.
- Leveraging second-order statistics (Gram matrices) to capture correlations across concatenated multi-layer features significantly enhances class separability.
- A brief initial adaptation phase (FSA) helps bridge the domain gap between pre-training and downstream tasks, improving overall performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.