Reproduce “DrivingDojo Dataset: Advancing Interactive and Knowledge-Enriched Driving World Model” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuqi Wang, Ke Cheng, Jiawei He, Qitai Wang, Hengchen Dai, Yuntao Chen, Fei Xia, Zhaoxiang Zhang
The DrivingDojo dataset and its accompanying Action Instruction Following (AIF) benchmark are introduced to advance driving world models by providing comprehensive ego-actions, diverse multi-agent interactions, and rich open-world knowledge. Models trained on DrivingDojo achieve higher visual quality and demonstrate superior fidelity to instructed actions, with mean absolute errors for in-domain longitudinal and lateral actions of 0.100m and 0.062m respectively.
Key insights
- Specifically curating a dataset for action diversity, multi-agent interactions, and rare open-world events is critical for training robust and generalizable driving world models.
- World models trained on such a comprehensive dataset can accurately generate future scenarios guided by various action instructions, effectively acting as action-controlled simulators.
- Evaluating world models with an Action Instruction Following benchmark provides a more functionally relevant assessment than visual quality metrics alone, demonstrating fidelity to intended maneuvers.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.