Reproduce “Putting Captions to the Test: Evaluating Video Caption Quality through Multiple-Choice Question Answering” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zizhen Wang, Bo Feng, Zhengfeng Lai, Shiyu Li, Yang Lu, Meng Cao, Ping Huang, Xiaoming Simon Wang
CapQuiz evaluates video-caption quality by testing whether a reader can answer human-verified multiple-choice questions about a video from its caption. Its scores distinguish incorrect claims from omitted details. In a study of 600 captions, CapF1 correlated with human overall quality at 0.643, versus 0.522 for a direct video-based judge. CapR covers benchmark questions, not all video content.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.