Reproduce “Overcoming the Stability Gap in Continual Learning” on OpenResearch

About this paper

Md Yousuf Harun, Christopher Kanan

Researchers from Rochester Institute of Technology and the University of Rochester introduced Stability Gap Mitigation (SGM), a method that addresses the temporary performance drop on old tasks (stability gap) during continual learning with large pre-trained deep neural networks. The SGM method achieved 20 times greater stability and a 16.7 times speedup in network updates compared to baseline continual learning and full retraining, respectively, while also maintaining or improving performance on the original pre-training dataset.

Key insights

• The stability gap is primarily caused by large initial loss signals for new classes and excessive network plasticity during updates to pre-trained models.
• Strategically reducing the initial loss via informed output unit initialization and dynamic soft targets, alongside controlled parameter updates via LoRA and OOCF, effectively mitigates the stability gap.
• Continual learning with pre-trained models can maintain or even improve performance on original pre-training datasets while efficiently acquiring new knowledge, offering substantial computational benefits over traditional retraining.

Summary by alphaXiv.