Reproduce “SAFE: Harnessing LLM for Scenario-Driven ADS Testing from Multimodal Crash Data” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Siwei Luo, Yang Zhang, Yao Deng, Linfeng Liang, Xi Zheng
The SAFE framework reconstructs autonomous driving test scenarios from multimodal crash data, leveraging large language models to process both textual summaries and visual sketches. It mitigates LLM hallucinations and improves scenario accuracy through Retrieval-Augmented Generation and self-validation, enhancing ADS bug detection and real-world crash reproduction rates.
Key insights
- Leveraging multimodal input (text and sketches) with LLMs, combined with structured reasoning and self-validation, significantly improves the accuracy and consistency of scenario reconstruction, reducing hallucination.
- A finely-grained Domain-Specific Language (DSL) is essential for precisely encoding complex road networks, traffic actor behaviors, and environmental contexts into high-fidelity scenarios.
- Context-aware prompt generation, which dynamically tailors prompts using a knowledge base, is critical for enabling LLMs to extract relevant and high-quality information from diverse crash reports.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.