Reproduce “Free Hunch: Denoiser Covariance Estimation for Diffusion Models Without Extra Costs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Severi Rissanen, Markus Heinonen, Arno Solin
Researchers at Aalto University developed "Free Hunch," a training-free method for efficiently and accurately estimating denoiser covariance in diffusion models, enhancing conditional generation for linear inverse problems. The approach consistently outperforms existing baselines, achieving a perceptual quality (LPIPS) of 0.317 for 15-step inpainting on ImageNet 256x256, a notable improvement over the next best baseline at 0.489.
Key insights
- Accurate, non-diagonal denoiser covariance estimation is crucial for unbiased conditional generation in diffusion models, preventing severe overestimation of guidance scales that often plagues simpler approximations.
- Valuable covariance information can be efficiently extracted from the diffusion process itself, combining prior data knowledge with local curvature observed during sampling, without needing extra training or architectural changes.
- Stochastic samplers can lead to more accurate covariance estimates than deterministic ones, as the inherent noise aids in more comprehensive exploration of the curvature space for BFGS updates.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.