Reproduce “Parallelizing Stochastic Gradient Descent for Least Squares Regression: mini-batching, averaging, and model misspecification” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Prateek Jain, Sham M. Kakade, Rahul Kidambi, Praneeth Netrapalli, Aaron Sidford

The authors analyze mini-batch stochastic gradient descent with tail-averaging for strongly convex streaming least-squares regression. Batches average gradients at one parameter value; later iterate averaging reduces noise. Up to a data-dependent threshold, they prove near-linear reductions in serial-update depth. In simulation, batch size 11 matches batch size 1 risk at about one-tenth the depth, not wall-clock time.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.