Reproduce “StyleVLA: Driving Style-Aware Vision Language Action Model for Autonomous Driving” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yuan Gao, Dengyuan Hua, Mattia Piccinini, Finn Rasmus Schäfer, Korbinian Moller, Lin Li, Johannes Betz

StyleVLA introduces a Vision Language Action model for autonomous driving that generates diverse and physically plausible trajectories based on natural language driving style instructions. The system, leveraging a new style-aware dataset and a physics-informed fine-tuning framework, enables a lightweight 4B-parameter model to achieve superior performance, scoring 0.55 in BEV and 0.51 in FPV scenarios, surpassing larger proprietary and state-of-the-art VLA models.

Key insights

  • Creating a dedicated, large-scale dataset with style-specific ground-truth trajectories and natural language instructions is critical for training personalized autonomous driving systems.
  • Incorporating a physics-informed kinematic consistency loss during VLM fine-tuning is crucial for generating physically plausible and accurate driving trajectories, moving beyond simple token prediction.
  • Lightweight, domain-adapted VLMs, when fine-tuned with appropriate data and physics-informed objectives, can significantly outperform larger, general-purpose foundation models in specialized autonomous driving tasks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.