Reproduce “A Benchmark and Multi-Agent System for Instruction-driven Cinematic Video Compilation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Peixuan Zhang, Chang Zhou, Ziyuan Zhang, Hualuo Liu, Chunjie Zhang, Jingqi Liu, Xiaohui Zhou, Xi Chen and 3 more
Researchers from multiple institutions introduced CineBench, a new benchmark for instruction-driven cinematic video compilation, and CineAgents, a multi-agent system that reformulates compilation into a "design-and-compose" paradigm. CineAgents outperformed existing baselines, achieving an F1-score of 64.13 and a compiled quality score of 9.01, while effectively addressing contextual collapse and temporal fragmentation in cinematic content.
Key insights
- Reframing cinematic video compilation into a "design-and-compose" paradigm, rather than simple "retrieve-and-rank," effectively addresses challenges like contextual collapse and temporal fragmentation.
- Constructing a hierarchical narrative memory (shot, event, story, and character levels) from source videos provides essential multi-level context for understanding complex cinematic narratives and interpreting instructions.
- An iterative narrative planning process, involving collaborative agents (Director and Orchestrator), allows for the logical refinement of a creative blueprint into a coherent compiled script, ensuring narrative flow.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.