Reproduce “MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xuan Ju, Yiming Gao, Zhaoyang Zhang, Ziyang Yuan, Xintao Wang, Ailing Zeng, Yu Xiong, Qiang Xu and 1 more
Researchers at ARC Lab, Tencent PCG, and The Chinese University of Hong Kong created MiraData, a large-scale video dataset featuring average 72.1-second durations and 318-word structured captions, alongside the MiraBench evaluation benchmark. This resource enables text-to-video generation models, such as their MiraDiT, to achieve improved motion strength, temporal consistency, and 3D consistency compared to prior datasets.
Key insights
- High-quality, long-duration videos with detailed structured captions are crucial for training generative models capable of producing advanced, temporally consistent, and controllable video content.
- Granular, multi-perspective captions, specifically covering main objects, backgrounds, camera movements, and video style, significantly enhance a model's ability to interpret complex prompts and generate dynamic, aligned videos.
- Novel evaluation metrics focusing on 3D consistency and object tracking strength are essential for accurately assessing the advanced capabilities of video generation models beyond traditional quality and alignment measures.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.