Reproduce “Dreaming the Unseen: World Model-regularized Diffusion Policy for Out-of-Distribution Robustness” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ziou Hu, Xiangtong Yao, Yuan Meng, Zhenshan Bing, Alois Knoll

The Dream Diffusion Policy (DDP) integrates a diffusion policy with a predictive diffusion world model to enhance robotic manipulation robustness under severe out-of-distribution (OOD) visual disturbances. This system dynamically shifts from relying on corrupted external vision to its internal predictive "imagination," achieving up to a 73.8% success rate on OOD MetaWorld tasks and an 83.3% average success rate across real-world OOD tasks, significantly outperforming existing baselines.

Key insights

  • Deep, co-optimized integration of a diffusion world model actively regularizes the diffusion policy, enabling it to generate actions from internally imagined latent states when real observations are corrupted.
  • The "real-imagination discrepancy" serves as an effective, intuitive detector for out-of-distribution events, allowing the system to switch from physical perception to internal prediction.
  • Recursive imagination, sustained by the world model's autoregressive forecasting, enables coherent and continuous control trajectories even in the complete absence of reliable external visual input.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.