Reproduce “Video-R1: Reinforcing Video Reasoning in MLLMs” on OpenResearch

About this paper

Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng, Junfei Wu, Xiaoying Zhang and 2 more

Researchers introduced Video-R1, the first framework to apply a rule-based reinforcement learning paradigm for enhancing temporal reasoning capabilities in Multimodal Large Language Models (MLLMs) for video. Video-R1, leveraging a new temporal-aware RL algorithm and dedicated video reasoning datasets, consistently outperforms previous state-of-the-art models, achieving 37.1% accuracy on VSI-Bench, surpassing GPT-4o's 34.0%.

Key insights

• Reinforcement Learning (RL) with a temporal-aware algorithm (T-GRPO) is crucial for unlocking generalized and robust video reasoning in MLLMs, offering performance improvements over SFT-only approaches.
• The explicit incentivization of temporal understanding through a contrastive reward mechanism effectively guides the MLLM to leverage correct temporal order, leading to deeper temporal reasoning rather than relying on superficial cues.
• Combining image-based reasoning data with video data during training provides a robust foundation of general reasoning skills that significantly boosts performance on complex video reasoning tasks.

Summary by alphaXiv.