Reproduce “Steering Dynamical Regimes of Diffusion Models by Breaking Detailed Balance” on OpenResearch

About this paper

Haiqi Lu, Ying Tang

Researchers at the University of Electronic Science and Technology of China introduced a method to accelerate generative diffusion models by incorporating a non-reversible anti-symmetric component into the forward drift dynamics. This approach accelerates the speciation transition, where models commit to distinct data modes, without altering the collapse transition, which governs memorization versus generalization.

Key insights

• Violating detailed balance through an anti-symmetric drift (Q) can accelerate the relaxation to equilibrium in diffusion models without altering the target stationary distribution.
• The speciation and collapse phase transitions in diffusion models are governed by distinct physical mechanisms, allowing for selective manipulation: speciation can be accelerated while collapse remains invariant.
• Optimal anti-symmetric perturbations can effectively decouple the beneficial acceleration of mode separation from the detrimental effects of memorization in generative processes.

Summary by alphaXiv.