Reproduce “NaviDriveVLM: Decoupling High-Level Reasoning and Motion Planning for Autonomous Driving” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ximeng Tao, Pardis Taghavi, Dimitar Filev, Reza Langari, Gaurav Pandey

The NaviDriveVLM framework addresses the trade-off between high-level reasoning and precise motion planning in VLM-based autonomous driving by proposing a decoupled architecture. It utilizes a frozen large VLM for semantic reasoning and a lightweight, fine-tuned VLM for waypoint prediction, achieving an overall L2 error of 0.46m on nuScenes while providing interpretable decision processes.

Key insights

  • Decoupling reasoning from planning allows leveraging the strong semantic understanding of large, frozen VLMs while achieving efficient and accurate control with lightweight, fine-tuned models.
  • The Navigator's output serves as an explicit and interpretable intermediate representation, enhancing transparency in decision-making and directly guiding the downstream planning module.
  • Explicit high-level commands significantly improve motion planning accuracy, demonstrating the importance of structured intention information in guiding autonomous driving behaviors.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.