Reproduce “Achieving More with Less: Additive Prompt Tuning for Rehearsal-Free Class-Incremental Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haoran Chen, Ping Wang, Zihan Zhou, Xu Zhang, Zuxuan Wu, Yu-Gang Jiang
Researchers from Fudan University developed Additive Prompt Tuning (APT), a rehearsal-free class-incremental learning method that applies additive prompts to the CLS token's attention in Vision Transformers. APT achieved state-of-the-art accuracy on major CIL benchmarks while significantly reducing computational overhead and trainable parameters by up to 99% compared to prior prompt-based approaches.
Key insights
- High performance in prompt-based CIL can be achieved with a minimalist additive prompt design, bypassing the computational burden of complex prompt selection and concatenation strategies.
- Targeting only the CLS token's attention with additive prompts is sufficient for capturing task-relevant features efficiently, leading to extreme parameter savings without sacrificing accuracy.
- A simple, inference-only fusion strategy for prompts effectively balances the retention of old knowledge with the adaptation to new tasks, simplifying the overall continual learning process.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.