Reproduce “On Learning Contrastive Representations for Learning with Noisy Labels” on OpenResearch

About this paper

Li Yi, Sheng Liu, Qi She, A. Ian McLeod, Boyu Wang

This work from the University of Western Ontario, NYU, and ByteDance presents Contrastive Regularization for Learning with Noisy Labels (CTRR), a method for training deep neural networks to learn robust representations from noisy data. The approach theoretically demonstrates how learned representations capture true label information while discarding corrupted label details, achieving state-of-the-art accuracy on various noisy benchmark and real-world datasets, with improvements of up to 30% over baselines in high noise settings.

Key insights

• Minimizing the proposed contrastive regularization is equivalent to maximizing mutual information, leading to representations that robustly capture true label information while effectively discarding corrupted label information.
• The `eL_ctr` function reverses the problematic gradient dominance, ensuring that correctly identified positive pairs have a greater impact on representation learning compared to noisy, incorrect pairs.
• Learned representations are robust against memorizing corrupted labels, yet they retain sufficient discriminative information to avoid underfitting and achieve accurate classification.

Summary by alphaXiv.