Reproduce “Timestep-Aware Block Masking for Efficient Diffusion Model Inference” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haodong He, Yuan Gao, Weizhong Zhang, Gui-Song Xia
An end-to-end framework dynamically optimizes which computational blocks to activate in pre-trained Diffusion Probabilistic Models at each denoising step. This approach enables up to a 2.75x inference speedup on LDM and 1.67x on DiT, while sustaining or enhancing generative quality and significantly reducing training memory requirements.
Key insights
- DPMs exhibit varying feature dynamics across denoising timesteps, implying that computational blocks can be selectively activated or skipped based on their local importance.
- A per-timestep optimization approach for acceleration masks can significantly reduce training memory overhead compared to global optimization, making it practical for complex DPMs.
- Dynamic adjustments to regularization coefficients based on timestep sensitivity and post-optimization mask rectification can further improve the speed-quality trade-off.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.