Reproduce “Antithetic Noise in Diffusion Models” on OpenResearch

About this paper

Jing Jia, Sifan Liu, Bowen Song, Wei Yuan, Liyue Shen, Guanyang Wang

The authors pair a diffusion model’s initial noise with its negation to generate images under the same condition. They apply antithetic Monte Carlo (AMC), averaging paired statistics to reduce uncertainty when paired statistics correlate negatively. For LSUN-Church pixel mean, confidence interval width is 1.64 with Monte Carlo and 0.16 with AMC at evaluation budgets; gains depend on the statistic.

Key insights

• Antithetic initial noise universally leads to strongly negatively correlated outputs in diffusion models, a property robust across different model types, datasets, and even extends to VAEs and Normalizing Flows.
• The learned score networks exhibit an approximate affine antisymmetry (ϵ_θ(t)(x) + ϵ_θ(t)(-x) ≈ 2c_t), providing a novel structural understanding of their behavior that aligns with theoretical expectations at high noise levels.
• The negative correlation between antithetic samples acts as an effective control variate, drastically reducing variance in Monte Carlo estimates, enabling more reliable statistical analysis without model retraining.

Summary by alphaXiv.