Reproduce “Uncertainty-Aware Distillation for Semi-Supervised Few-Shot Class-Incremental Learning” on OpenResearch

About this paper

Yawen Cui, Wanxia Deng, Haoyu Chen, Li Liu

The UaD-CE framework, an end-to-end semi-supervised learning approach for Few-Shot Class-Incremental Learning (FSCIL), effectively leverages unlabeled data to enhance model adaptability, achieving up to 4 percentage points higher final overall accuracy compared to state-of-the-art FSCIL methods. It integrates class-balanced self-training and uncertainty-aware knowledge distillation to mitigate overfitting on novel classes and catastrophic forgetting.

Key insights

• Class-balanced pseudo-label generation effectively mitigates classification bias by strategically selecting unlabeled samples, preventing dominance of 'easy-to-classify' categories.
• Filtering exemplar sets based on prediction uncertainty improves the reliability of knowledge distillation, enhancing the preservation of old knowledge.
• An adaptive weighting scheme for distillation loss dynamically balances the retention of old knowledge with the learning of new categories, optimizing overall performance.

Summary by alphaXiv.