Reproduce “Taming Teacher Forcing for Masked Autoregressive Video Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Deyu Zhou, Quan Sun, Yuang Peng, Kun Yan, Runpei Dong, Duomin Wang, Zheng Ge, Nan Duan and 3 more

The MAGI framework enhances autoregressive video generation by introducing Complete Teacher Forcing (CTF) to eliminate the training-inference gap and incorporating dynamic interval and noise injection strategies to mitigate exposure bias. This approach achieves state-of-the-art FVD on Kinetics-600 and UCF-101 among AR models, generating over 100 coherent frames with linear inference scaling.

Key insights

  • Aligning training and inference conditions through Complete Teacher Forcing is crucial for capturing temporal motion and ensuring video coherence, outperforming Masked Teacher Forcing which struggles with motion modeling despite potentially better static frame quality.
  • Dynamic interval training and dynamic noise injection synergistically enhance the model's robustness and generalization capabilities, allowing it to generate stable and coherent long video sequences.
  • Frame-level autoregressive generation, when combined with efficient KV Cache utilization, offers a practical balance between high-quality output and computational efficiency for scaling to extended video lengths.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.