Reproduce “STORM-Bench: Evaluating Online Video QA under Evolving and Incomplete Evidence” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Siru Zhong, Shenghan Tan, Rihong Yan, Xiaohui Lv, Yuzheng Zhuang, Shuai Tao, Wulong Liu, Haohuan Fu and 1 more
STORM-Bench evaluates online video question-answering from causal video prefixes, testing both task answers and whether evidence supports answering. It uses change-intensity and answerability labels across five real egocentric domains and two simulations. Molmo2-8B and Qwen3.5-9B achieve nearly identical accuracy (60.31% and 60.33%), but their STORM-BR scores differ (35.57% versus 28.07%). This shows accuracy can miss reliability differences, without identifying why.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.