Reproduce “Patch-Based Diffusion Models Beat Whole-Image Models for Mismatched Distribution Inverse Problems” on OpenResearch

About this paper

Jason Hu, Bowen Song, Jeffrey A. Fessler, Liyue Shen

This research develops a patch-based diffusion model framework to effectively solve image inverse problems under conditions of mismatched training data or severe data scarcity. The approach demonstrates superior robustness to overfitting and out-of-distribution shifts, yielding higher quality reconstructions compared to whole-image diffusion models in challenging scenarios.

Key insights

• Patch-based diffusion priors exhibit stronger generalizability than whole-image priors in data-scarce or OOD settings, as local image statistics tend to be more transferable across different image distributions.
• Self-supervised refinement of a patch-based diffusion model, even from a single measurement, enables effective adaptation of an OOD prior to complex inverse problems, achieving performance comparable to or exceeding models trained on perfectly matched, large datasets.
• Patch-based models demonstrate inherent resistance to overfitting and memorization when fine-tuned on very small datasets, maintaining stable and high reconstruction performance where whole-image models tend to degrade rapidly.

Summary by alphaXiv.