Reproduce “Boosting Discriminative Visual Representation Learning with Scenario-Agnostic Mixup” on OpenResearch

About this paper

Siyuan Li, Zicheng Liu, Zedong Wang, Di Wu, Zihan Liu, Stan Z. Li

Scenario-Agnostic Mixup (SAMix) presents a unified framework for learned data augmentation that extends effectively to both supervised and self-supervised visual representation learning, outperforming prior mixup methods across various benchmarks while also offering a computationally efficient pre-trained variant.

Key insights

• Optimizing mixup generation requires balancing local smoothness between mixed samples with global discrimination from other data points in the manifold.
• A novel Mixer architecture, featuring non-linear content modeling and adaptive lambda encoding, effectively prevents 'trivial solutions' (e.g., uninformative masks) in learned mixup, particularly in self-supervised learning.
• A mixup generator can be effectively pre-trained on a large dataset and then reused for downstream tasks, significantly reducing computational cost and improving practical applicability.

Summary by alphaXiv.