Reproduce “DriveMRP: Enhancing Vision-Language Models with Synthetic Motion Data for Motion Risk Prediction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhiyi Hou, Enhui Ma, Fang Li, Zhiyi Lai, Kalok Ho, Zhanqian Wu, Lijun Zhou, Long Chen and 6 more
DriveMRP enhances Vision-Language Models for motion risk prediction in autonomous driving by introducing a synthetic high-risk motion data generation pipeline and a VLM-agnostic agent with visual prompting. The framework achieves an 88.03% accident recognition accuracy on synthetic data, up from 27.13% for baselines, and exhibits robust zero-shot generalization to real-world high-risk scenarios, increasing accuracy from 29.42% to 68.50%.
Key insights
- Synthetic data generation, when carefully curated and validated (e.g., with Human-in-the-Loop), can effectively address the challenge of data scarcity for high-risk autonomous driving scenarios.
- Converting abstract numerical trajectory data into a visual prompt significantly improves VLM's spatial reasoning and risk prediction capabilities by aligning with their visual processing strengths.
- Integrating global BEV context with ego-view imagery and employing a 'chain-of-thought' reasoning process enables VLMs to provide both accurate risk predictions and interpretable causal explanations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.