Reproduce “Token Perturbation Guidance for Diffusion Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Javad Rajabi, Soroush Mehraban, Seyedmorteza Sadat, Babak Taati
Researchers from the University of Toronto, Vector Institute, and ETH Zürich introduced Token Perturbation Guidance (TPG), a training-free and condition-agnostic method that enhances diffusion model output by perturbing intermediate token representations. TPG achieves state-of-the-art performance for unconditional generation, nearly doubling FID improvement over baselines, and matches Classifier-Free Guidance effectiveness in conditional scenarios without requiring model retraining.
Key insights
- Effective guidance can be achieved by disrupting local patterns in intermediate token representations via shuffling while preserving global statistics and their $\ell_2$-norm.
- TPG's guidance signal emulates key beneficial characteristics of CFG, such as maintaining near-orthogonality to the true noise and exhibiting high guidance magnitude, especially in lower frequency bands during early denoising steps.
- Perturbing downsampling layers specifically yields the most effective guidance, indicating the importance of guiding global structure early in the denoising process.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.