Reproduce “Grouping First, Attending Smartly: Training-Free Acceleration for Diffusion Transformers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sucheng Ren, Qihang Yu, Ju He, Alan Yuille, Liang-Chieh Chen
Researchers from Johns Hopkins University and collaborators introduce GRAT (GRouping first, ATtending smartly), a training-free method to accelerate Diffusion Transformer inference. The method achieves up to a 9.2x end-to-end speedup for 8192x8192 image generation with Flux and up to a 15.8x attention speedup for video generation with HunyuanVideo, all while preserving or improving the original generative quality.
Key insights
- Attention in pretrained Diffusion Transformers is empirically observed to be predominantly sparse and local, indicating redundancy in global attention computations.
- Designing attention mechanisms with hardware-aware grouping and structured access patterns significantly improves real-world GPU utilization and throughput, leading to substantial speedups over theoretical FLOPs reductions.
- A balance between local context and selective long-range dependencies, as seen in GRAT-X, can maintain or even enhance generative quality while offering considerable speed improvements.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.