Reproduce “Dynamically Expanding Capacity of Autonomous Driving with Near-Miss Focused Training Framework” on OpenResearch

About this paper

Ziyuan Yang, Zhaoyang Li, Jianming Hu, Yi Zhang

Researchers from Tsinghua University developed a framework that generates 'near-miss' scenarios for autonomous vehicle training by applying Robust Adversarial Reinforcement Learning. This approach uses novel reward functions and alternating training to create challenging yet recoverable situations, resulting in autonomous vehicles that demonstrate improved handling of complex, safety-critical interactions while maintaining low collision severity.

Key insights

• Applying the 'Zone of Proximal Development' principle enables optimal learning for autonomous vehicles by generating challenges slightly beyond current capabilities but still achievable.
• A novel background vehicle reward function based on 'Maximum Impulse' and 'Obstacle Frames' guides the generation of mild, learnable near-misses instead of catastrophic collisions.
• Alternating training phases allow both autonomous vehicles and background vehicles to continuously adapt, creating a dynamic curriculum that matches scenario difficulty to the AV's evolving skills.

Summary by alphaXiv.