Reproduce “Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like process” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Guy Blanc, Neha Gupta, Gregory Valiant, Paul Valiant
The authors study how independent label perturbations during stochastic gradient descent (SGD) favor simpler fits among zero-training-error settings. Near such solutions, they characterize drift toward lower average squared prediction gradients on training inputs, measuring parameter sensitivity. In a two-layer one-dimensional ReLU example, curve length is about 5.2 without noise and 4.3 with it; this is not a generalization guarantee.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.