Reproduce “Gaussian and Non-Gaussian Universality of Data Augmentation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kevin Han Huang, Peter Orbanz, Morgane Austern
The paper analyzes how dependence among augmented examples affects estimator uncertainty and prediction risk. It treats transformations from each observation as a block, replacing each block with a mean- and covariance-matched surrogate under smoothness and noise-stability conditions. In an oracle ridgeless model with k=5, augmentation softens risk near d/n=1 but creates another peak near d/n=5, a model-specific result.
Key insights
- Despite strong dependencies, noise-stable augmented estimators exhibit universality, where their asymptotic behavior is determined primarily by the first two moments (mean and covariance) of the augmented observations.
- Data augmentation's effects on estimator variance and regularization are conditional; it can increase uncertainty for non-linear estimators and its regularization in high dimensions depends critically on the interplay of sample size, dimension, and number of augmentations.
- The impact of augmentation is not solely tied to distributional invariance; for linear estimators, maintaining or reducing the second moment of data under transformations is key for variance reduction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.