Reproduce “VideoMAR: Autoregressive Video Generatio with Continuous Tokens” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hu Yu, Biao Gong, Hangjie Yuan, DanDan Zheng, Weilong Chai, Jingdong Chen, Kecheng Zheng, Feng Zhao
Researchers at the University of Science and Technology of China developed VideoMAR, an autoregressive image-to-video model that employs continuous tokens and integrates frame-by-frame temporal generation with spatial masked prediction. The model achieves state-of-the-art performance among autoregressive methods, outperforming prior models like Cosmos with substantially fewer parameters and training data, while also exhibiting training-free spatial and temporal extrapolation capabilities.
Key insights
- Operating with continuous tokens, rather than discrete, significantly reduces information loss and improves visual quality in autoregressive video generation.
- A combined frame-wise causal and spatial masked generation approach effectively captures temporal dependencies and intra-frame correlations in video.
- Curriculum learning strategies for sequence length and resolution, alongside a progressive temperature sampling, are essential for training stable and high-quality long-duration video models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.