Reproduce “Competing with the Empirical Risk Minimizer in a Single Pass” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Roy Frostig, Rong Ge, Sham M. Kakade, Aaron Sidford

Streaming SVRG estimates a reference gradient from a batch and corrects fresh stochastic gradients using each sample’s gradient there. Under regularity assumptions, it uses one pass and small memory to approach the empirical risk minimizer’s population excess-risk rate. For least-squares regression, a finite-sample bound approaches the ERM benchmark, while initial-error dependence decreases at arbitrarily high polynomial order beyond a stated sample threshold.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.