Reproduce “VideoTG-R1: Boosting Video Temporal Grounding via Curriculum Reinforcement Learning on Reflected Boundary Annotations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lu Dong, Haiyu Zhang, Han Lin, Ziang Yan, Xiangyu Zeng, Hongjie Zhang, Yifei Huang, Yi Wang and 3 more
VideoTG-R1 significantly improves video temporal grounding performance for multimodal large language models by introducing a curriculum reinforcement learning framework and reflected boundary annotations. This approach achieves state-of-the-art results across several benchmarks, utilizing only 10% of the training samples and 21% of the computational budget of full-data training.
Key insights
- Filtering partially annotated samples using MLLM-based self-reflection significantly improves data quality and training efficiency, outperforming other data selection strategies.
- Progressively easing the difficulty for hard-to-ground samples through curriculum learning and dynamic masking leads to more stable and effective reinforcement learning for MLLMs.
- The observed gains from these strategies are more pronounced under supervised fine-tuning (SFT) compared to GRPO, suggesting GRPO's inherent robustness to ambiguous and challenging samples.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.