Reproduce “On Error Propagation of Diffusion Models” on OpenResearch

About this paper

Yangming Li, Mihaela van der Schaar

A theoretical framework by researchers at the University of Cambridge establishes that Diffusion Models inherently suffer from error propagation across their sequential denoising steps. Their proposed regularization method, incorporating a cumulative error term estimated via MMD and optimized using a bootstrapping technique, reduces this error and improves image generation quality, lowering FID scores on CIFAR-10 by 18.83% (from 3.61 to 2.93) and on ImageNet by 29.55% (from 3.62 to 2.55).

Key insights

• Diffusion Models inherently suffer from error propagation, where errors accumulate and amplify across sequential denoising steps, a phenomenon rigorously proven through a derived propagation inequality.
• The cumulative error, measurable by Maximum Mean Discrepancy (MMD), empirically increases rapidly as denoising progresses towards the final sample, directly correlating with degraded generation quality.
• Regularizing Diffusion Models to explicitly reduce this cumulative error, particularly at earlier denoising steps, effectively makes denoising modules robust to input inaccuracies and mitigates error propagation.

Summary by alphaXiv.