Reproduce “Dynamic Orthogonal Continual Fine-tuning for Mitigating Catastrophic Forgettings” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zhixin Zhang, Zeming Wei, Meng Sun

Researchers at Peking University developed Dynamic Orthogonal Continual (DOC) fine-tuning, a regularization-based method for Large Language Models that mitigates catastrophic forgetting by dynamically tracking and orthogonally projecting evolving functional directions. This approach demonstrated superior average accuracy and reduced backward transfer compared to prior methods across various continual learning benchmarks, including long task sequences.

Key insights

  • The 'functional directions' of historical tasks, used by regularization-based continual learning methods, experience significant drift in the parameter space during sequential fine-tuning, rendering fixed representations ineffective.
  • Dynamically tracking these functional directions through Online PCA, even without access to historical data, is critical for maintaining their relevance and preventing catastrophic forgetting.
  • Orthogonally constraining new task updates relative to these dynamically tracked historical functional directions effectively prevents interference and preserves knowledge in a computationally efficient manner.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.