Reproduce “Consistency Model is an Effective Posterior Sample Approximation for Diffusion Inverse Solvers” on OpenResearch

About this paper

Tongda Xu, Ziran Zhu, Jian Li, Dailan He, Yuanyuan Wang, Ming Sun, Ling Li, Hongwei Qin and 3 more

A new approach improves Diffusion Inverse Solvers (DIS) by using Consistency Models (CMs) to generate valid posterior samples, proving particularly effective when the forward operator is a neural network. This leads to superior performance for complex inverse problems such as semantic segmentation and image captioning, and enables a more efficient CM inversion method.

Key insights

• Posterior mean approximations commonly used in Diffusion Inverse Solvers often generate samples outside the natural image distribution's support, causing performance degradation, especially with neural network operators.
• Solutions from Probability Flow Ordinary Differential Equations (PF-ODEs) inherently produce in-distribution posterior samples, offering a theoretically sound alternative for Diffusion Inverse Solvers.
• Consistency Models (CMs) effectively distill PF-ODEs, enabling efficient and high-quality in-distribution posterior sample generation for iterative inverse problem solving.
• Adding a small amount of Gaussian noise to the Consistency Model output during inversion enhances robustness against overfitting the forward operator and adversarial examples.

Summary by alphaXiv.