Reproduce “Staying on the Manifold: Geometry-Aware Noise Injection” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Albert Kjøller Jacobsen, Johanna Marie Gegenfurtner, Georgios Arvanitidis
Researchers at the Technical University of Denmark introduced geometry-aware noise injection techniques that augment data by respecting its intrinsic manifold structure, rather than applying perturbations in the ambient space. These methods, including projected tangent space, geodesic, and intrinsic Brownian motion noise, consistently improved model generalization and robustness on complex synthetic manifolds and for image classification on autoencoder-approximated manifolds.
Key insights
- Projected tangential noise acts as an implicit regularizer that specifically penalizes only the tangential component of the learned function's gradient, encouraging flatness along the data manifold where variations are semantically meaningful.
- Perturbations generated along the data manifold produce more meaningful augmented samples that preserve the underlying data structure, as shown by 'digit-looking' images produced by geometry-aware noise on MNIST.
- Geometry-aware noise methods offer improved robustness to the noise intensity hyperparameter, maintaining stable or better performance across a wider range of values compared to ambient noise, particularly on complex manifolds.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.