Reproduce “MinD: Learning A Dual-System World Model for Real-Time Planning and Implicit Risk Analysis” on OpenResearch

About this paper

Xiaowei Chi, Kuangzhi Ge, Jiaming Liu, Siyuan Zhou, Peidong Jia, Zichen He, Yuzhen Liu, Tingguang Li and 4 more

MinD (Manipulate in Dream) introduces a dual-system world model that integrates video generative models into real-time robotic control by using computationally inexpensive, single-step latent predictions for planning. The framework achieves state-of-the-art performance on RL-Bench with a 63.0% success rate and in real-world manipulation tasks with a Franka robot at 72.50% average success, operating at 11.3 FPS while enabling implicit risk assessment.

Key insights

• Robotic policies can effectively guide actions using computationally cheap, single-step latent representations of future states, eliminating the need for fully denoised, high-fidelity future video frames.
• An asynchronous co-training strategy with a diffusion-forcing alignment loss can successfully bridge and synchronize distinct generative models operating at different frequencies for unified visual imagination and action control.
• Latent features from a world model, even from early denoising steps, can implicitly encode uncertainty and potential task failures, providing a valuable signal for real-time risk assessment without explicit supervision.

Summary by alphaXiv.