Reproduce “Distribution Backtracking Builds A Faster Convergence Trajectory for Diffusion Distillation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shengyuan Zhang, Ling Yang, Zejian Li, An Zhao, Chenye Meng, Changyuan Yang, Guang Yang, Zhiyuan Yang and 1 more
Distribution Backtracking Distillation improves one-step diffusion generation by replacing direct endpoint matching with a sequence of intermediate score-based targets. Across unconditional and conditional image-generation benchmarks, it lowers FID and accelerates convergence—for example, reaching 1.38 FID on ImageNet64 with one network evaluation and converging up to 5.67× faster than a comparable baseline on CIFAR-10.
Key insights
- Intermediate targets reduce teacher–student score mismatch, particularly when the student is far from the teacher distribution and at low noise levels.
- A moderate number of trajectory checkpoints is useful, but overly dense checkpointing or excessive degradation can make the targets ineffective and reduce generation quality.
- The method treats diffusion distillation as a path-following optimization problem rather than solely as direct matching to a final distribution.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.