Reproduce “Last Iterate Risk Bounds of SGD with Decaying Stepsize for Overparameterized Linear Regression” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jingfeng Wu, Difan Zou, Vladimir Braverman, Quanquan Gu, Sham M. Kakade

The authors analyze last-iterate stochastic gradient descent for overparameterized linear regression with a tail geometric step-size schedule. They bound excess risk through covariance-spectrum-dependent effective dimension, separating bias from noise variance. Under stated assumptions, variance is bounded above and below by constant multiples of sigma-squared effective dimension over K. These results concern linear regression, and bias bounds are not shown tight.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.