Reproduce “Fewer Denoising Steps or Cheaper Per-Step Inference: Towards Compute-Optimal Diffusion Model Deployment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhenbang Du, Yonggan Fu, Lifu Wang, Jiayi Qian, Xiao Luo, Yingyan
The Energy-Efficient, Intelligent, and Reliable Computing (EIC) Lab at Georgia Institute of Technology developed PostDiff, a training-free framework for compute-optimal diffusion model deployment. This framework reduces computational costs and latency for pre-trained diffusion models by combining mixed-resolution denoising and a hybrid module caching strategy, achieving up to 63.4% FLOPs reduction while improving FID by 1.8 on SD V1.5 and demonstrating broad applicability across various models.
Key insights
- For a relatively high generation fidelity, reducing per-step inference cost via methods like PostDiff is generally more effective than simply reducing the number of denoising steps.
- Mixed-resolution denoising provides a "win-win" scenario, not only reducing FLOPs but also improving generation fidelity by enhancing low-frequency components during early denoising stages.
- Feature maps, particularly within attention layers, and text guidance from cross-attention exhibit high similarity across consecutive denoising steps, making them prime targets for training-free caching and reuse.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.