Reproduce “Learning from Mistakes: Post-Training for Driving VLA with Takeover Data” on OpenResearch

About this paper

Yinfeng Gao, Deqing Liu, Qichao Zhang, Yupeng Zheng, Haochen Tian, Guang Li, Hangjun Ye, Long Chen and 2 more

TakeVLA, an iterative post-training framework, enhances Vision-Language-Action (VLA) models for autonomous driving by enabling proactive learning from error-prone situations and active policy exploration. It achieved a Driving Score of 89.72 and a Success Rate of 73.73% on the Bench2Drive benchmark, surpassing other end-to-end methods and improving average Time-to-Collision (TTC) by 0.43 seconds.

Key insights

• Pre-takeover language supervision effectively trains VLA models to anticipate hazards earlier, significantly enlarging safety margins and shifting from reactive error correction to proactive hazard anticipation.
• Active exploration via Scenario Dreaming enables the VLA model to discover and refine more optimal driving policies beyond passive imitation of expert demonstrations.
• The iterative nature of TakeVLA, combining SFT with pre-takeover data and RFT through Scenario Dreaming, consistently improves closed-loop driving performance and safety metrics.

Summary by alphaXiv.