Reproduce “Cache Me if You Can: Accelerating Diffusion Models through Block Caching” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Felix Wimbauer, Bichen Wu, Edgar Schoenfeld, Xiaoliang Dai, Ji Hou, Zijian He, Artsiom Sanakoyeu, Peizhao Zhang and 6 more
Researchers from Meta GenAI and academic institutions developed Block Caching, a method to accelerate diffusion model inference by reusing internal U-Net block computations. The technique achieves 1.5x-1.8x speedup while simultaneously yielding improved image quality (lower FID scores) compared to non-cached baselines operating at the same computational budget.
Key insights
- Internal layer blocks within diffusion model U-Nets, particularly `SpatialTransformer` blocks, exhibit predictable and often redundant computational behavior across denoising steps, which can be leveraged for acceleration.
- An adaptive caching schedule based on step-to-step output changes allows for significant speedup without substantial quality degradation.
- Minor misalignments from caching can be effectively corrected with a lightweight, distilled scale-shift adjustment, preserving or even enhancing image quality.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.