Reproduce “AdaGrad under heavy-tailed noise: symmetry, asymmetry and damping” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hanyu Yang
The authors analyze AdaGrad under heavy-tailed noisy gradients, comparing conditionally symmetric noise with general unbiased noise. AdaGrad scales updates using an accumulator that includes the current gradient, allowing asymmetric noise to bias steps. At p=3/2, worst-case rate exponents for the expected average gradient norm are 1/12 generally and 1/6 under symmetry. Damping can recover the symmetric rate; these are constructed bounds, not empirical results.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.