Reproduce “Score-Based Generative Modeling with Critically-Damped Langevin Diffusion” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Tim Dockhorn, Arash Vahdat, Karsten Kreis

The authors introduce critically-damped Langevin diffusion (CLD) for score-based image generation, coupling data to an auxiliary velocity and adding noise only to velocity. The model learns a velocity-conditional score, while a symmetric splitting sampler improves CIFAR-10 FID to 3.07 from 7.00 at 150 network evaluations. CLD-SGM reports FID 2.23 versus 2.41 for DDPM++; findings are limited to tested settings.

Key insights

  • Augmenting the state space with a critically-damped velocity variable simplifies the target score function, making the learning task easier for the neural network.
  • The neural network primarily learns the score function of the conditional velocity given data (`∇_v log p(v_t|x_t)`), which is intrinsically smoother and closer to a Normal distribution's score.
  • Symmetric operator splitting techniques from molecular dynamics can be effectively applied to construct more accurate and efficient SDE solvers for generative diffusion processes.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.