Reproduce “VibeVideoBench: Evaluating Multimodal Agents on Code-Driven Video Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Muxi Diao, Wenbin Xu, Lele Yang, Xuyang Ma, Yuzhe Chen, Jiankang Chen, Kaiyu Jiang, Changyi Liu and 16 more
VibeVideoBench evaluates agents that create complete videos by writing code for every frame and sound. It tests nine systems on 239 tasks across 12 categories in one-hour runs. Claude Opus 5.5 achieved strict success on 76.3% of tasks, meaning each had a passed core check and no failed core checks; this measures requirements, not video quality or delivery.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.