Reproduce “Diffusion models as plug-and-play priors” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Alexandros Graikos, Esmeralda S. Whitammer, Nebojsa Jojic, Dimitris Samaras

Independently trained, unconditional Denoising Diffusion Probabilistic Models (DDPMs) can function as versatile plug-and-play priors for inference when combined with auxiliary differentiable constraints. This approach facilitates conditional image generation, robust semantic segmentation with domain transfer, and continuous relaxation for combinatorial optimization tasks like TSP, achieving competitive results.

Key insights

  • Independently trained, unconditional DDPMs can effectively function as fixed, plug-and-play priors to guide inference when combined with differentiable constraints, without needing fine-tuning.
  • The proposed free energy minimization framework enables versatile conditional generation and inference by combining a pre-trained generative prior with diverse, independently defined differentiable constraints.
  • Leveraging DDPMs to reason over continuous relaxations of combinatorial problems within an image space introduces a unique visual inductive bias, reminiscent of human puzzle-solving.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.