Reproduce “Scaling Cross-Embodiment World Models for Dexterous Manipulation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zihao He, Bo Ai, Tongzhou Mu, Yulin Liu, Weikang Wan, Jiawei Fu, Yilun Du, Henrik I. Christensen and 1 more

Researchers from UC San Diego and collaborators developed a particle-based world model for dexterous manipulation that unifies state and action representations, allowing a single model to learn from heterogeneous robot and human interaction data. This approach enables zero-shot transfer and model-predictive control on diverse robotic hands, significantly outperforming human-only models in real-world deformable object reshaping tasks.

Key insights

  • Transferable knowledge for cross-embodiment learning resides in the physical interactions induced by manipulators, rather than their embodiment-specific joint spaces, which can be captured by particle-based world models.
  • An embodiment scaling law exists, where increasing the diversity of training embodiments consistently reduces prediction error and improves generalization to unseen robot hands.
  • Co-training with both simulated robot and real human data, especially at an approximately 1:1 ratio, provides complementary benefits by broadening interaction distributions and grounding the model in physical reality.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.