Reproduce “EvoVid: Temporal-Centric Self-Evolution for Video Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shiqi Huang, Ziyue Wang, Zhongrong Zuo, Han Qiu, Qi She, Bihan Wen
EvoVid introduces a self-evolving framework for Video Large Language Models, enabling them to improve video understanding and reasoning directly from raw, unannotated videos. The approach uses temporal-centric reward signals for question generation and answer grounding, yielding consistent performance gains across benchmarks, such as a 19.3% improvement on TempCompass for the Qwen2.5-VL-3B model, and outperforming supervised fine-tuning in some comparisons.
Key insights
- Explicitly integrating temporal sensitivity into the reward functions of a self-evolving framework effectively guides Video-LLMs to acquire complex temporal reasoning skills without human annotation.
- The co-evolution of a temporal-aware Questioner (generating more challenging and temporally dependent questions) and a temporal-grounded Solver (improving its ability to answer and temporally localize) leads to sustained performance enhancements.
- The dual approach of using temporal perturbation for question generation and temporal segment grounding for answer verification are complementary strategies that foster autonomous temporal understanding in video models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.