Reproduce “Worldscape-MoE: A Unified Mixture-of-Experts World Model for Scalable Heterogeneous Action Control” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jianjie Fang, Yongyan Xu, Ziyou Wang, Chen Gao, Yuchao Huang, Zhaolu Wang, Rongze Tang, Mingyuan Jia and 8 more

Researchers from Tsinghua University and Manifold AI developed Worldscape-MoE, a unified Mixture-of-Experts world model, to integrate diverse action controls like camera trajectories, robot joint actions, and hand motions into a single framework. The model improves physical consistency and temporal coherence, achieving state-of-the-art performance across locomotion, manipulation, and hand-motion tasks while demonstrating strong generalization capabilities.

Key insights

  • Different control types represent distinct interfaces to the same underlying world, necessitating a factorization into shared world dynamics and control-specific action interpretation.
  • The Mixture-of-Experts (MoE) architecture enables structured capacity allocation, allowing a shared expert to capture common world dynamics while dedicated experts specialize in modality-specific action semantics, avoiding interference.
  • A unified learning framework, especially with MoE, can absorb heterogeneous supervision without sacrificing individual control performance, transforming fragmented datasets into a cumulative training resource.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.