Reproduce “VL-DPO: Vision-Language-Guided Finetuning for Preference-Aligned Autonomous Driving” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhefan Xu, Ghassen Jerfel, Marina Haliem, Qi Zhao, Jeonhyung Kang, Khaled S. Refaat
VL-DPO introduces a framework that finetunes autonomous driving motion forecasting models using preference pairs generated by a frozen, zero-shot Vision-Language Model (VLM). The method achieves superior alignment with human driving preferences, showing an 11.94% increase in Rater Feedback Score over the pretrained baseline and competitive geometric accuracy.
Key insights
- A frozen, zero-shot VLM can effectively serve as a scalable and high-quality proxy for human driving preferences by autonomously generating explicit comparative signals for model finetuning.
- Combining imitation learning with VLM-guided DPO offers a powerful, complementary finetuning strategy, leveraging human positive examples and VLM-generated negative (what to avoid) signals for more robust preference alignment.
- Modular integration of the VLM as an offline critic enables superior preference alignment without incurring real-time inference latency or catastrophic forgetting in the deployed motion forecasting model.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.