Reproduce “SRIL: Selective Regularization for Class-Incremental Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jisu Han, Jaemin Na, Wonjun Hwang
Researchers at Ajou University developed Selective Regularization for Class-Incremental Learning (SRIL), an integrated deep learning approach that employs gradient-based feature distillation and confidence-aware weight interpolation to mitigate catastrophic forgetting. This method dynamically balances retaining old knowledge with acquiring new information, leading to state-of-the-art average accuracy on CIFAR-100 (up to 1.96 percentage points higher) and ImageNet-Subset (up to 1.73 percentage points higher).
Key insights
- Asymmetric feature distillation, guided by gradient similarity, can effectively manage the integration of new information while preserving existing patterns, inspired by cognitive pattern completion and separation.
- Dynamically adjusting the degree of weight interpolation based on the model's performance on old data allows for intelligent adaptation, only enforcing stability when necessary and permitting exploratory learning otherwise.
- Combining selective feature regularization with adaptive weight interpolation creates a synergistic approach that outperforms static regularization schemes in addressing the stability-plasticity dilemma.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.