Reproduce “DriveDreamer-Policy: A Geometry-Grounded World-Action Model for Unified Generation and Planning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yang Zhou, Xiaofeng Wang, Hao Shao, Letian Wang, Guosheng Zhao, Jiangnan Shao, Jiagang Zhu, Tingdong Yu and 3 more
DriveDreamer-Policy tests whether driving planning can use explicit geometry alongside imagined future video. A language model processes instructions, multi-view images, and action context; depth informs video generation, and both depth and video inform trajectory generation. Its combined depth-video-action variant scores 89.2 PDMS on Navsim v1, versus 88.9 for video-plus-action in an ablation, not a real-world driving result.
Key insights
- Explicit geometric grounding, achieved through dedicated depth map generation, serves as an effective 3D scaffold that significantly improves the coherence and informativeness of future video predictions.
- Jointly modeling depth, video, and actions within a single, modular architecture, guided by an LLM, leads to more robust and informed motion planning decisions.
- The causal conditioning pathway and modular design allow for flexible operating modes, such as planning-only or imagination-enabled planning, while maintaining controllable latency for practical deployment.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.