Reproduce “A Flat Minima Perspective on Understanding Augmentations and Model Robustness” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Weebum Yoo, Sung Whan Yoon
This study presents a unified theoretical framework explaining how label-preserving data augmentations enhance model robustness against general distribution shifts by demonstrating that these augmentations promote flatter minima in the loss landscape. The approach leads to tighter generalization bounds, a finding consistently supported by empirical evaluations across various benchmarks and augmentation techniques.
Key insights
- A mathematical duality exists between perturbations in input space (augmentations) and perturbations in parameter space, implying that input augmentations can be functionally compensated by changes in model weights.
- Augmentations satisfying the Proximal-Support Augmentation (PSA) condition actively steer model optimization towards finding flatter minima in the loss landscape, indicating their role in shaping the energy surface.
- Flatter minima, induced by PSA-compliant augmentations, are shown to lead to tighter PAC-Bayesian generalization bounds, thereby enhancing model robustness against a wide array of distributional shifts.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.