Reproduce “Are LLMs The Way Forward? A Case Study on LLM-Guided Reinforcement Learning for Decentralized Autonomous Driving” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Timur Anvar, Jeffrey Chen, Yuyan Wang, Rohan Chandra

Researchers conducted a case study evaluating the integration of small, locally deployable Large Language Models (LLMs) with Reinforcement Learning (RL) for decentralized autonomous driving. They found that LLM-guided reward shaping enhanced collision avoidance in RL agents, achieving success rates up to 100% in certain highway scenarios, but often introduced a conservative bias that reduced driving efficiency compared to RL-only methods.

Key insights

  • Small, locally deployable LLMs can provide effective semantic signals for reward shaping in RL, improving collision avoidance, but inherently introduce a conservative bias towards safety over efficiency.
  • Directly using LLMs as autonomous driving policies leads to extremely conservative behavior, severely degrading efficiency and exploiting simulator characteristics, making them unsuitable for real-time control.
  • The specific LLM chosen and the reward shaping integration method critically influence the resulting driving policy and its trade-offs between safety, efficiency, and maneuverability.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.