Reproduce “ProphetDWM: A Driving World Model for Rolling Out Future Actions and Videos” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiaodong Wang, Peixi Peng
Researchers from Peking University and Peng Cheng Laboratory developed ProphetDWM, an end-to-end driving world model that jointly predicts future driving videos and low-level actions. The model achieves superior performance, reducing FID to 6.9 and FVD to 190.5 for video generation, and an average L1 error of 0.0915 for action prediction on the Nuscenes dataset.
Key insights
- Joint training of action and video prediction modules with latent actions significantly improves video quality and action consistency compared to decoupled or frozen approaches.
- The proposed multi-scale fusion module for state context integration in the diffusion model backbone leads to faster convergence and better video quality.
- ProphetDWM effectively models latent action dynamics, allowing for long-term action and video prediction without requiring full future action sequences as input.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.