Reproduce “Towards Error-Free Long Video Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Shuning Chang, Weihua Chen, Jiasheng Tang, Hao Xu, Zeyu Zhang, Hangjie Yuan, Yu Lu, Ruigang Niu and 3 more

The Zhejiang University and Alibaba Group team developed a framework for generating high-quality, dynamic, and identity-consistent long videos, integrating diffusion models with an autoregressive mechanism. Their approach, featuring causal attention with KV caching and a Truncation-Rectified Flow (T-RFlow) technique, achieves superior performance on benchmarks like VBench, notably improving subject consistency to 0.9457 and image quality to 0.6620 for minute-level videos while maintaining coherence for durations up to three minutes.

Key insights

  • Errors in rectified flow denoising accumulate from distinct frequency components at different timesteps: low-frequency issues at early steps and high-frequency issues at late steps.
  • Long-term context, beyond local frames, is crucial for preventing attribute drift and maintaining identity consistency in extended video generation.
  • Modifying the denoising process by truncating specific rectified flow integration steps can effectively suppress error accumulation without requiring additional training.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.