Reproduce “Work Zones challenge VLM Trajectory Planning: Toward Mitigation and Robust Autonomous Driving” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yifan Liao, Zhen Sun, Xiaoyun Qiu, Zixiao Zhao, Wenbing Tang, Xinlei He, Xinhu Zheng, Tianwei Zhang and 2 more
Research from the Hong Kong University of Science and Technology (Guangzhou) and Nanjing University of Aeronautics and Astronautics systematically reveals that Vision Language Models (VLMs) fail in 68.0% of work zone trajectory planning scenarios. Their proposed REACT-Drive framework reduces the average displacement error by 3x and achieves real-time inference at 0.58s per scene, improving autonomous driving safety.
Key insights
- Mainstream VLMs demonstrate a high failure rate of 68.0% in work zone trajectory planning, with specific patterns like "dead end roads" and "turning through work zones" being particularly challenging.
- A novel methodology involving scene graph mining, clustering, and human verification successfully identified 8 distinct and recurring VLM failure patterns in work zone scenarios.
- Combining the general reasoning of VLMs with a structured, verifiable knowledge base of constraint rules via RAG can effectively mitigate VLM failures in safety-critical, long-tail scenarios.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.