Reproduce “T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kaiyue Sun, Kaiyi Huang, Xian Liu, Yue Wu, Zihan Xu, Zhenguo Li, Xihui Liu
Introducing T2V-CompBench, the first comprehensive benchmark designed for evaluating compositional Text-to-Video generation, which features 1400 diverse prompts and specialized metrics. An extensive evaluation reveals existing T2V models face considerable challenges in synthesizing videos with complex compositional attributes and precise spatio-temporal dynamics.
Key insights
- Evaluating compositionality in Text-to-Video generation is significantly more complex than in Text-to-Image due to the added challenge of maintaining spatio-temporal dynamics consistently over time.
- Specialized evaluation metrics, integrating MLLM, object detection, and tracking capabilities, are essential for accurately measuring fine-grained compositional aspects, unlike general video metrics.
- The established benchmark, building on prior work in compositional T2I evaluation, provides a standardized framework to systematically diagnose limitations in current T2V models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.