Reproduce “PromptFusion: Decoupling Stability and Plasticity for Continual Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haoran Chen, Zuxuan Wu, Xintong Han, Menglin Jia, Yu-Gang Jiang
The PromptFusion framework addresses the stability-plasticity dilemma in continual learning by decoupling these objectives into specialized prompt-tuning modules. It achieves state-of-the-art performance, including a 5.3% higher average accuracy on memory-free Split-Imagenet-R, while an efficient variant (PromptFusion-Lite) reduces computational overhead by 19.2%.
Key insights
- The stability-plasticity dilemma in continual learning can be effectively resolved by decoupling these two conflicting objectives into independent, specialized modules.
- Different prompt tuning methods exhibit distinct continual learning properties: CoOp excels in stability and robustness to complex datasets, while VPT shows higher plasticity and proficiency in learning new information.
- A dynamic fusion of these specialized prompt tuning modules, supported by a frozen pre-trained backbone, enables superior performance across diverse continual learning scenarios.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.