Reproduce “Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yilun Du, Conor Durkan, Robin Strudel, Joshua B. Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet and 1 more
The authors propose composing pretrained diffusion models by replacing ordinary reverse diffusion with MCMC transitions across progressively lower noise levels. An energy-based parameterization supports mixture composition and Metropolis-corrected sampling. In a 2D product test, score-model U-HMC reaches RAISE 2.52 versus 1.55 for reverse diffusion, but MCMC can take five times longer and products without overlap can fail.
Key insights
- Standard reverse diffusion sampling incorrectly samples from composed distributions at intermediate noise levels, causing compositional generation failures; the learned model is not the issue, but the sampler is.
- Diffusion models can be re-interpreted as Energy-Based Models (EBMs) whose training approximates score matching, enabling the use of MCMC for correct sampling from complex, composed distributions.
- An explicit energy-based parameterization for diffusion models facilitates Metropolis-corrected MCMC sampling and enables new compositional operators like mixtures and negation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.