Reproduce “ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring” on OpenResearch

About this paper

David Berthelot, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Kihyuk Sohn, Han Zhang, Colin Raffel

ReMixMatch presents an advanced semi-supervised learning algorithm that significantly enhances accuracy and data efficiency by integrating distribution alignment and augmentation anchoring with an adaptive augmentation strategy called CTAugment. This method achieves state-of-the-art performance on image classification benchmarks, enabling high accuracy with up to 16 times fewer labeled examples compared to previous approaches.

Key insights

• Aligning the model's average predictions on unlabeled data with the known class distribution significantly stabilizes and improves semi-supervised learning, leveraging a lesser-explored aspect of mutual information maximization.
• Decoupling the target generation (via weak augmentation) from the regularization of strong augmentations provides a robust way to harness the benefits of powerful data augmentations for consistency training.
• An adaptive, online approach to learning augmentation policies, as implemented in CTAugment, is crucial for effectively utilizing strong augmentations in severely data-limited semi-supervised settings.

Summary by alphaXiv.