Reproduce “Agent-driven Long-tail Simulation for Autonomous Driving” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Junru Gu, Lijin Yang, Jianing Huang, Shu Liu, Zhongzhan Huang, Hang Zhao

Researchers from Tsinghua University and Bosch Research developed an agent-driven simulation framework for autonomous driving that leverages large language models to control surrounding road participants, creating diverse and interactive long-tail scenarios. Their new SemanticPlan benchmark reveals that even state-of-the-art planners achieve limited safety scores in these complex environments, while an IDM+LLM planner showed improved semantic progress.

Key insights

  • Leveraging instruction-following LLMs and Vision-Language Models (VLMs) enables the generation of diverse, intentional, and reactive behaviors for non-ego agents in AD simulations, enhancing realism beyond rule-based methods.
  • Offloading kinematic and feasibility constraints to the simulator's native route planner significantly improves the physical plausibility and robustness of LLM-controlled vehicle agents compared to having the agent model directly generate low-level motions.
  • The SemanticPlan benchmark consistently reveals that current state-of-the-art AD planners struggle to achieve high safety scores and effective task completion in complex, interactive, and semantically rich long-tail scenarios, highlighting critical performance gaps.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.