Reproduce “AIMMerging: Adaptive Iterative Model Merging Using Training Trajectories for Language Model Continual Learning” on OpenResearch

About this paper

Yujie Feng, Jian Li, Xiaoyu Dong, Pengfei Xu, Xiaohui Zhou, Yujia Zhang, Zexin LU, Yasha Wang and 3 more

AIMMerging introduces an adaptive iterative model merging framework for continual learning in Large Language Models (LLMs), dynamically scheduling merges by monitoring learning and forgetting signals from training trajectories. The method achieves state-of-the-art performance, with an average relative improvement of 80% on Forward Transfer and 59% on Backward Transfer over existing methods, effectively balancing new knowledge acquisition and catastrophic forgetting across various LLM scales.

Key insights

• Dynamically monitoring internal training trajectories provides crucial, real-time signals about a model's learning and forgetting status, enabling intelligent merging decisions.
• An adaptive merging strategy, which adjusts merging frequency and timing based on these signals, is more effective and efficient than fixed-interval merging for balancing continual learning objectives.
• Explicitly capturing and adaptively fusing both new knowledge (from current task training) and historical knowledge (from brief fine-tuning on a memory buffer) optimally addresses the trade-off between knowledge transfer and catastrophic forgetting.

Summary by alphaXiv.