Reproduce “Tutorial on Diffusion Models for Imaging and Vision” on OpenResearch

About this paper

Stanley H. Chan

Stanley Chan's tutorial from Purdue University provides a comprehensive, mathematically rigorous guide to diffusion models, detailing their principles, derivations, and connections to statistical physics like the Langevin and Fokker-Planck equations. It unifies disparate model types by showing their correspondence to Stochastic Differential Equations (SDEs).

Key insights

• Denoising Diffusion Probabilistic Models (DDPMs) and Score-Matching Langevin Dynamics (SMLDs) are unified under a common framework as discrete-time approximations of continuous-time Stochastic Differential Equations (SDEs).
• The Evidence Lower Bound (ELBO) serves as the core objective for training diffusion models, with DDPM training effectively simplifying to a noise prediction task.
• Learning the 'Stein's score function' (the gradient of the log-likelihood) via denoising score matching enables effective sampling from complex data distributions using Langevin dynamics, bypassing direct density estimation.

Summary by alphaXiv.