Reproduce “Non Gaussian Denoising Diffusion Models” on OpenResearch

About this paper

Eliya Nachmani, Robin San Roman, Lior Wolf

Researchers introduced and formalized the use of non-Gaussian noise distributions, such as Mixture of Gaussians and Gamma distributions, within Denoising Diffusion Probabilistic Models. This approach significantly improved generation quality and efficiency across image and speech synthesis tasks, with Mixture of Gaussians achieving an FID of 31.21 compared to 299.71 for Gaussian DDPMs with 10 inference steps on CelebA.

Key insights

• The empirical noise patterns observed in pre-trained Gaussian Denoising Diffusion Probabilistic Models often deviate from pure Gaussian distributions.
• Non-Gaussian noise distributions, specifically Mixture of Gaussians and Gamma distributions, can be rigorously integrated into the DDPM framework while preserving the crucial closed-form sampling property for efficient generation.
• Utilizing noise distributions with greater degrees of freedom can lead to substantial improvements in the generation quality and computational efficiency of diffusion models across different data modalities.

Summary by alphaXiv.