Reproduce “Measuring the local shape of learning-rate transfer curves” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Satoki Ishikawa, Rio Yokota, Ryo Karakida
The authors analyze how trained loss changes with learning rate, aiming to characterize sensitivity across model widths. They present EoS-LR, derived under edge-of-stability and constrained-minimum assumptions, and FP-LR, a time-averaged fixed-point heuristic. In tested settings, both estimates track finite-difference measurements, while averaging improves FP-LR agreement; the work does not demonstrate faster hyperparameter search.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.