Reproduce “LOVE-R1: Advancing Long Video Understanding with an Adaptive Zoom-in Mechanism via Multi-Step Reasoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shenghao Fu, Qize Yang, Yuan-Ming Li, Xihan Wei, Xiaohua Xie, Wei-Shi Zheng
LOVE-R1 introduces an adaptive zoom-in mechanism powered by multi-step reasoning to enhance long video understanding in Large Video-Language Models, effectively balancing temporal context with fine-grained spatial details. This method achieves an average of 3.1% point performance improvement across four major long video benchmarks, outperforming existing reasoning and non-reasoning models.
Key insights
- Long video understanding can be effectively modeled as an adaptive multi-step reasoning process, where an LVLM intelligently requests high-resolution visual information only when necessary.
- Explicitly optimizing the model's "zoom-in" capability through decoupled reinforcement finetuning, which provides process-specific rewards, is crucial for robust temporal grounding and superior performance.
- A "slow-fast" video template (appending high-resolution clips to a low-resolution overview) effectively balances computational constraints with the need for both global temporal context and local spatial detail.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.