Reproduce “Fine-Grained Knowledge Selection and Restoration for Non-Exemplar Class Incremental Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiang-Tian Zhai, Xialei Liu, Lu Yu, Ming-Ming Cheng
This paper presents a framework for Non-Exemplar Class Incremental Learning (NECIL) that employs Vision Transformers and fine-grained knowledge management. It achieves state-of-the-art accuracy on CIFAR100, TinyImageNet, and ImageNet-Subset datasets by mitigating catastrophic forgetting and classifier bias.
Key insights
- The inherent patch-level representations of Vision Transformers can be effectively leveraged to implement fine-grained, adaptive knowledge distillation, balancing plasticity and stability.
- Regulating the distribution of prototype offsets across tasks can lead to more accurate restoration of old class prototypes, directly addressing and reducing classifier bias.
- Applying different regularization strategies to task-relevant and task-agnostic regions significantly improves incremental learning performance compared to uniform knowledge distillation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.