Reproduce “CAST-Bench: Tracing Every Sound to Its Source in Multi-Shot Audio-Video Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yucong Liu, Qi Tian, Shuyuan Tu, Kaihang Pan, Weijie Kong, Chao Pan

CAST-BENCH evaluates generated audiovisual soundtracks by pairing prompts with annotations specifying requested events, sources, shots, and timing. It assesses these bindings across 900 prompts in nine sound-source categories, reporting single- and multi-shot results separately. All nine shared models score lower on multi-shot prompts; Seedance-2.0 scores 78.03 single-shot and 72.15 multi-shot, but the weighted overall scores use different metrics and weights.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.