Reproduce “DynamicRad: Content-Adaptive Sparse Attention for Long Video Diffusion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yongji Long, Shijun Liang, Jintao Li, Yun Li
DYNAMICRAD introduces a content-adaptive sparse attention framework for long video diffusion, which integrates structured sparsity with dynamic, content-aware selection. This approach delivers 1.7 x to 2.5 x inference speedups and over 80% effective sparsity while preserving or enhancing video quality and temporal coherence.
Key insights
- Self-attention in video diffusion exhibits spatiotemporal locality, allowing for efficient structured sparse masks that decay with distance.
- Content-adaptive sparse attention can be implemented with minimal runtime overhead by using offline optimization to determine sparsity parameters and a lightweight semantic motion router at inference.
- Thoughtful sparsity, particularly the dynamic mode, can sometimes improve video quality by selectively pruning irrelevant tokens, contributing to a "denoising" effect in extended sequences.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.