Reproduce “Settling the Burn-in Gap: Sharp Risk Bounds for Tail-Averaged SGD in Linear Regression” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Qingyue Zhao, Xuheng Li, Quanquan Gu
The authors analyze constant-stepsize online SGD with tail averaging for linear regression. They show burn-in damps initial error in fast directions while fluctuations transfer energy into slow directions that persist. Under a spectral-feedback condition, upper and lower risk bounds match up to constants under stated assumptions. A two-dimensional example yields a polynomial risk lower bound, unlike an exponentially smaller lower bound.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.