Reproduce “A Journey to the Edge of Stability” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jaerin Lee, Kyoung Mu Lee
The authors examine how optimizers shape training before and at the edge of stability. They compare learning rates using effective step h = ηQ(1) to account for each optimizer’s gradient gain and filtering. In large-batch CIFAR-10 runs, trajectories are similar before the edge but diverge near it; predicted and measured final sharpness align, with R² = 0.9983 in the tested MLP setup.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.