Reproduce “From Slow Bidirectional to Fast Autoregressive Video Diffusion Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tianwei Yin, Qiang Zhang, Richard Zhang, William T. Freeman, Fredo Durand, Eli Shechtman, Xun Huang
CausVid generates interactive video by producing successive chunks from past context and distilling a bidirectional diffusion teacher into a causal student using four denoising steps. On a 10-second, 120-frame test, it reports 1.3-second latency and 9.4 frames per second. Its VBench-Long total is 84.27, highest reported in the table, but extremely long videos still show quality degradation.
Key insights
- Knowledge distillation from a powerful bidirectional teacher can effectively mitigate error accumulation in an autoregressive student, preserving quality over long sequences.
- Distribution Matching Distillation (DMD) can be successfully adapted to the temporal domain of video generation to accelerate models without significant quality loss.
- Combining a causal architecture with efficient inference techniques like KV caching enables interactive frame rates and low latency for video diffusion models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.