Reproduce “World Action Models are Zero-shot Policies” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng, Shenyuan Gao, Sihyun Yu, George Kurian, Suneel Indupuru, You Liang Tan and 28 more
DreamZero learns robot control by jointly predicting future video and action, using a pretrained video model alongside language, visual context, and robot state. On ten unseen AgiBot G1 tasks, it achieved 39.5% average task progress, versus 16.3% for the best pretrained VLA baseline trained on the same robot data. This partial-progress result does not show universal mastery.
Key insights
- Jointly predicting future world states (video) and actions with a video diffusion model provides powerful spatiotemporal priors, enabling zero-shot generalization to unseen physical motions and environments.
- Generalist policies can be learned effectively from diverse, non-repetitive robot data, reducing the reliance on laborious, repetitive demonstrations.
- Cross-embodiment transfer is highly efficient with WAMs, allowing learning from video-only demonstrations and few-shot adaptation to new robot platforms, crucial for scaling with human video data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.