Reproduce “Stochastic Autoregressive Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ilan Doron-Arad, Idan Mehalel, Elchanan Mossel
The paper studies learning in stochastic autoregressive generation under base, CoT, and end-to-end supervision. CoT reveals trajectories, while end-to-end training reveals only final tokens; both target final-token probabilities with squared loss. General guarantees require accuracy shifts. A constructed finite class requires end-to-end sample complexity scaling as M/ε times CoT complexity, a worst-case result rather than evidence about language models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.