Reproduce “V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shulin Tian, Minglun Li, Yuhao Dong, Hao Ding, Jiarui Yao, Haiwen Diao, Jingkang Yang, Hongyuan Zhu and 1 more
A framework for Vision-Language Models, V-Rubrics, employs rubric-based reinforcement learning to enhance visual faithfulness by providing fine-grained, item-level rewards. This method improves grounded reasoning, particularly for complex academic and visual math tasks, by addressing credit assignment failures in multimodal responses.
Key insights
- Visual faithfulness can be effectively reformulated as a fine-grained credit-assignment problem, allowing for targeted supervision signals during VLM post-training.
- Item-level, structured rubrics provide more granular and effective feedback for multimodal reasoning than traditional scalar rewards, enabling models to learn specific visual facts and reasoning steps.
- Localizing reward advantages to specific response prefixes significantly enhances learning by connecting specific rubric fulfillments or violations directly to the corresponding generated tokens.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.