Reproduce “Discrete generative diffusion models without stochastic differential equations: a tensor network approach” on OpenResearch

About this paper

Luke Causer, Grant M. Rotskoff, Juan P. Garrahan

A framework for discrete generative diffusion models uses tensor networks to exactly represent noising and denoising dynamics, eliminating the need for stochastic differential equations or learned score functions. This method facilitates efficient and accurate sampling of complex discrete lattice systems, demonstrating high MCMC acceptance rates and successful capture of multiple modes in models like the 2D Ising.

Key insights

• Tensor networks provide an exact and efficient framework for discrete diffusion models, representing distributions as MPS and evolution operators as MPOs, thereby avoiding SDEs and score function learning.
• The "connected update" MCMC protocol, which correlates noising and denoising steps, achieves higher acceptance rates and better sample decorrelation compared to conventional disconnected updates.
• Dynamically adjusting the denoising time (T) during learning is vital for maintaining high MCMC acceptance while ensuring adequate sample decorrelation.

Summary by alphaXiv.