Reproduce “PCR: Proxy-based Contrastive Replay for Online Class-Incremental Continual Learning” on OpenResearch

About this paper

Huiwei Lin, Baoquan Zhang, Shanshan Feng, Xutao Li, Yunming Ye

The paper introduces Proxy-based Contrastive Replay (PCR), a novel approach for Online Class-Incremental Continual Learning that combines the stability of proxy-based methods with the balanced learning of contrastive approaches. This method significantly reduces catastrophic forgetting and improves knowledge retention by modifying the contrastive loss to use stable class proxies and manage gradient propagation more effectively, resulting in consistent performance gains across benchmark datasets.

Key insights

• Limiting the contrastive loss denominator to only proxies of classes present in the current batch helps mitigate the bias issue where new classes can unduly affect old class proxies.
• Using stable class proxies instead of individual samples in contrastive calculations enhances training stability, particularly with small batch sizes common in continual learning.
• The method generates more uniform gradients across all class proxies, directly addressing the problem of imbalanced gradient propagation seen in other approaches.

Summary by alphaXiv.