Reproduce “Adaptive Memory Replay for Continual Learning” on OpenResearch

About this paper

James Seale Smith, Lazar Valkov, Shaunak Halbe, Vyshnavi Gutta, Rogerio Feris, Zsolt Kira, Leonid Karlinsky

An adaptive memory replay framework is introduced for continual learning in large Foundation Models, which selectively replays past data exhibiting maximum forgetting using a bandit-based approach. This "zero-cost" protocol reduced final loss and forgetting rates compared to standard rehearsal on vision (e.g., DomainNet final loss 23.65% vs 30.41%) and language tasks (LLaMA 7B final loss 12.88% vs 44.11%) without increasing overall computational cost.

Key insights

• For large Foundation Models, the primary constraint in continual learning shifts from memory capacity to computational cost, emphasizing the need for efficient data selection over storage.
• Dynamically identifying and replaying past samples that the model is actively 'forgetting' the most, conditioned on the current task, is more effective than uniform or static sampling strategies.
• Integrating sophisticated continual learning capabilities into large-scale models without increasing overall training time is feasible by strategically managing computational resources through methods like discarding new task data.

Summary by alphaXiv.