Reproduce “WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Driving” on OpenResearch

About this paper

Xuerun Yan, Zhexi Lian, Nuoheng Zhang, Shiyu Fang, Haoran Wang, Chen Lv, Jia Hu, Binyang Song

Researchers from Tongji University and Nanyang Technological University introduce WCog-VLA, a dual-level world-cognitive Vision-Language-Action model that integrates semantic forecasting with generative physical evolution for end-to-end autonomous driving. The framework enables proactive driving and strategic social reasoning, achieving a state-of-the-art Predictive Driving Model Score (PDMS) of 92.9 on NAVSIM v1 and an Extended PDMS (EPDMS) of 85.9 on NAVSIM v2.

Key insights

• Integrating semantic-level world forecasting with generative-level physical evolution within a dual-level cognition framework is crucial for shifting from reactive to proactive autonomous driving.
• Explicitly modeling 3D spatial awareness and decoupling VLM outputs for distinct cognitive and reasoning roles enhances environmental understanding and strategic social planning.
• A decoupled diffusion transformer (ADDT) with representation alignment efficiently synthesizes high-fidelity multi-agent trajectories, offering significant inference speedup while maintaining robustness to denoising steps.
• Supervising the VLM with game-theoretic Chain-of-Thought (Game-CoT) annotations allows the model to learn 'if-what' imagination, enabling strategic social reasoning for safer and more efficient interactions.

Summary by alphaXiv.