Reproduce “VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang and 4 more

VLM-R1 uses reinforcement learning to train vision-language models for referring-expression grounding and open-vocabulary object detection, scoring outputs against annotated boxes with task-specific rewards. In experiments with Qwen2.5-VL-3B, a length-penalized detection reward raised overall OVDEval NMS-AP to 31.01, while grounding accuracy on LISA-Grounding reached 63.14 versus 54.82 for SFT; gains remain task- and reward-dependent.

Key insights

  • R1-style reinforcement learning enhances VLM generalization to out-of-domain visual reasoning tasks, moving beyond supervised fine-tuning's tendency for memorization.
  • Careful reward engineering, exemplified by the 'odLength' reward, is crucial to mitigate 'reward hacking' and align model behavior with true task objectives in visual RL.
  • RL can foster emergent reasoning behaviors, such as the 'OD aha moment' where the VLM spontaneously adopts a two-step thinking process for complex object detection.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.