Reproduce “Code2Video-Compass: A Comprehensive Benchmark for General-Purpose Code-Based Video Generation across T2V, I2V, and V2V” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zhenghan Chen, Zekai Shao, Xin Lin, Leting Cheng, Siming Chen

Code2Video-Compass evaluates whether multimodal models can generate executable code for complete videos from text briefs and optional image or video references. It covers 2,104 tasks across 10 creative categories and evaluates 15 models using objective and subjective metrics. Performance generally worsens as construction, animation, and state-dependency complexity rises; results come from single-call generations without execution feedback or repair.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.