Reproduce “Reinforcement Learning Tuning for VideoLLMs: Reward Design and Data Efficiency” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hongyu Li, Songhao Han, Yue Liao, Junfeng Luo, Jialin Gao, Shuicheng Yan, Si Liu
Temporal-RLT is a reinforcement learning tuning framework for VideoLLMs that enhances video-specific reasoning by employing dual reward functions for semantic and temporal understanding and a variance-aware data selection strategy. The framework improves temporal grounding on Charades-STA by 14.0 mIoU and ActivityNet by 14.7 mIoU, while maintaining strong performance on VideoQA tasks.
Key insights
- Reinforcement learning, particularly GRPO with carefully designed reward functions, can effectively enhance video-specific semantic and temporal reasoning in LLMs.
- Enforcing a structured reasoning trace through output tags (<think>, <answer>) improves interpretability and provides valuable signals for reward computation.
- Data efficiency in RLT can be achieved by prioritizing samples that exhibit higher variance in model predictions, indicating a better learning signal.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.