Reproduce “DriveReward: A Comprehensive Dataset and Generative Vision-Language Reward Model for Autonomous Driving” on OpenResearch

About this paper

Qimao Chen, Fang Li, Yuechen Luo, Zehan Zhang, Haiyang Sun, Fangzhen Li, Bing Wang, Guang Chen and 6 more

DriveReward introduces a comprehensive dataset and a 1-billion parameter Vision-Language Reward Model (DriveReward-1B) designed to evaluate multi-modal autonomous driving trajectories. The model demonstrates improved performance in reinforcement learning fine-tuning and trajectory selection by providing multi-dimensional assessments covering safety, command following, and legality.

Key insights

• Specialized Vision-Language Models, when combined with domain-specific pre-training and task-specific fine-tuning on diverse, annotated trajectories, can effectively bridge the domain gap for autonomous driving reward modeling.
• Integrating a reasoning Chain-of-Thought (CoT) and an explicit geometry grounding module significantly enhances a VLM's ability to interpret complex visual contexts and accurately evaluate multi-dimensional driving trajectories.
• Augmenting datasets with synthesized counterfactual driving behaviors and novel evaluation dimensions like Command Following and Legality is crucial for developing robust reward models capable of assessing both nominal and anomalous scenarios.

Summary by alphaXiv.