Reproduce “Squeezed Diffusion Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jyotirmai Singh, Samar Khanna, James Burgess
Researchers at Stanford University developed Squeezed Diffusion Models (SDM) that modify noise injection based on data's principal components, demonstrating that mild 'antisqueezing' along the principal axis improves generative quality, achieving up to a 15% FID reduction on CIFAR-10 without architectural changes.
Key insights
- Optimal generative performance is achieved with mild 'antisqueezing' (increasing noise variance along the principal data axis), counter-intuitively opposing the initial hypothesis inspired by quantum metrology's 'squeezing'.
- The standard SDM, which only scales the principal direction, performs comparably to or better than the HDM, suggesting that strict uncertainty product conservation is not essential for improvements.
- Antisqueezing primarily boosts recall, indicating that this noise shaping encourages the model to generate a wider diversity of high-quality samples.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.