Reproduce “Adaptive Gradient Descent without Descent” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yura Malitsky, Konstantin Mishchenko
AdGD sets gradient-descent step sizes using successive gradients, without function values or line search. Each update is an ordinary gradient step, limited by a growth cap and an estimate of local curvature from changes in iterates and gradients. For convex functions with locally Lipschitz gradients, the authors prove convergence and an O(1/k) objective-gap bound for weighted averages, not final iterates.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.