Reproduce “Gumbel Straight Flow: Distilling Autoregressive Models into One-step Flow Maps” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yeongmin Kim, Arnaud Doucet, Andrew Campbell, Valentin De Bortoli, Thomas Mensink, David Ruhe

GSF uses an autoregressive teacher’s Gumbel-Max sampling noise to pair noise with generated tokens and train a parallel flow-map student along linear paths. The paper proves these teacher-induced paths do not intersect before the endpoint and are straight for an ideal flow-matching model with sufficient capacity. At one-step OpenWebText generation, GSF reports GM 0.94 and perplexity 114.9, but the autoregressive teacher remains better.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.