Reproduce “Are We Overthinking One-Step Generation?” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Woobin Park, Jaeyeon Kim, Yilun Du, Sitan Chen, Bohyung Han
Priming Models train a one-step image generator by first reconstructing clean images from corrupted inputs, then matching generated and real examples in frozen feature space. On ImageNet, Priming-L reaches FID 1.19 in one network pass using 238.95 EFLOPs, versus Drifting-L’s FID 1.61 using 1,476.39 EFLOPs. This is about 6.2 times less reported training compute for this comparison only.
Key insights
- Reconstruction pretraining is a preparation for distributional refinement, not the final generative objective: high-quality samples emerge during post-training.
- The amount of retained signal in the corrupted input matters; ImageNet results were strongest at noise level 0.8, while fully noisy inputs performed poorly.
- The authors’ Jacobian-alignment analysis is consistent with the idea that pretraining helps the model respond to distributional correction directions, though it does not establish this as the sole cause.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.