Reproduce “Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Driving” on OpenResearch

About this paper

Aron Distelzweig, Faris Janjoš, Andreas Look, Anna Rothenhäusler, Daniel Jost, Oliver Scheel, Raghu Rajan, Daphne Cornelisse and 2 more

Researchers from the University of Freiburg, Bosch Center for AI, Coburg University, and NYU introduced BehaviorBench, a comprehensive test suite connecting large-scale Reinforcement Learning policies to established autonomous driving benchmarks. This framework features challenging, interactive scenarios and diverse traffic agents, demonstrating that a hybrid PDM+PPO planner consistently outperforms baselines, while scaling RL alone does not guarantee generalization across varied agent behaviors.

Key insights

• Scaling Reinforcement Learning (RL) alone through increased agent steps does not inherently induce generalization across different traffic agent behaviors; policies can overfit to self-play opponents.
• Hybrid planning approaches, combining the safety of rule-based systems (like PDM) with the assertiveness of learned policies (like PPO), offer superior and more robust performance in complex, interactive scenarios.
• Existing benchmarks' reliance on simplistic scenarios and uniform traffic models significantly limits their ability to accurately assess autonomous driving policies' true capabilities and generalization.

Summary by alphaXiv.