Reproduce “VideoResearcher: Self-Improving Tool Design for Long-Video Understanding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dingqiang Ye, Dongdi Zhao, Kaishen Wang, Qingqiao Hu, Jingchen Sun, Yijun Liang, Yuqi Jia, Yiqiao Huang and 6 more
VideoResearcher develops executable video-inspection tools from an agent’s failed evidence searches, while keeping its Worker and perception model fixed. Its Solving loop answers questions, and its Evolving loop diagnoses missing capabilities, builds and tests candidate tools, and retains only accuracy-improving changes. Across three benchmarks, average Overall accuracy rose from 72.1% to 74.5% in the reported common-base comparison.
Key insights
- Tool-level adaptation can improve long-video understanding without updating the worker’s parameters, reasoning policy, or underlying perception model, isolating evidence acquisition as the optimization target.
- Successful tools tend to combine broad temporal localization with focused inspection of short candidate windows; plausible alternatives can still hurt performance and are rejected through empirical verification.
- Improvements are not monotonic across evolution rounds, indicating diminishing returns and a risk of development-set overfitting when newly generated tools are repeatedly selected on the same questions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.