Reproduce “DiffVLA: Vision-Language Guided Diffusion Planning for Autonomous Driving” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Anqing Jiang, Yu Gao, Zhigang Sun, Yiru Wang, Jijun Wang, Jinghao Chai, Qian Cao, Yuweng Heng and 6 more

DiffVLA introduces an end-to-end autonomous driving system that integrates a hybrid sparse-dense perception module, a vision-language model for high-level guidance, and a diffusion-based planning module. The system achieved an Extended Per-Drive Metric Score (EPDMS) of 45.007 in the challenging Navsim-v2 competition, demonstrating its ability to generate diverse and robust trajectories in reactive environments.

Key insights

  • Combining sparse and dense perception efficiently captures both explicit instance-level details and implicit scene context for autonomous driving.
  • Diffusion models effectively capture the multi-modality of driving actions, allowing for diverse and plausible trajectory generation.
  • Vision-Language Models can provide crucial high-level semantic guidance, improving contextual understanding and decision-making for autonomous driving systems.
  • The post-processing step of applying a small deceleration to predicted trajectories can effectively mitigate collision risks at high speeds, indicating a practical calibration method for real-world application.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.