Reproduce “Putting a Face to Forgetting: Continual Learning meets Mechanistic Interpretability” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sergi Masip, Gido M. van de Ven, Javier Ferrando, Tinne Tuytelaars
Researchers from KU Leuven and the University of Groningen introduced a mechanistic framework that explains catastrophic forgetting in continual learning as geometric transformations of individual feature vectors within a neural network. This framework identifies two primary mechanisms of forgetting: capacity degradation (due to feature overlap or fading) and readout misalignment, demonstrating their dynamics in both a tractable model and a Vision Transformer trained on a standard benchmark.
Key insights
- Forgetting can be mechanistically understood as either capacity degradation (reduced exclusivity or magnitude of feature encoding) or readout misalignment (impaired extraction of feature information by downstream computations).
- A U-shaped relationship exists between task similarity and forgetting, with intermediate similarity leading to greater interference than either very similar or very dissimilar tasks.
- Network depth exacerbates forgetting, and increasing the number of readout probes amplifies capacity degradation by reducing orthogonal space for features.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.