Reproduce “Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Mehdi Sajjadi, Mehran Javanmardi, Tolga Tasdizen

Researchers at the University of Utah developed a deep semi-supervised learning regularization method that enforces prediction consistency across stochastic transformations and network perturbations. This approach, combining a Transformation/Stability Loss with a Mutual-Exclusivity Loss, reduced error rates on MNIST from 5.44% to 0.55% with only 100 labels and achieved new state-of-the-art results on CIFAR10 (3.00%) and CIFAR100 (21.43%).

Key insights

  • The intrinsic stochasticity within modern ConvNet training (e.g., data augmentation, dropout, random pooling) can be directly harnessed as an effective regularization mechanism for semi-supervised learning.
  • Enforcing consistency in network predictions under varying stochastic conditions significantly enhances model generalization and robustness, especially when labeled data is scarce.
  • Combining a consistency-based regularization with a mutual exclusivity objective effectively guides the learning process to produce more stable and well-separated decision boundaries in the feature space.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.