Reproduce “SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model” on OpenResearch

About this paper

Qiayuan Liao, Yiyang Shao, Junfeng Long, Kevin Zakka, Roberto Horowitz, Pieter Abbeel, Koushil Sreenath

SuperMimic trains a humanoid controller to track reference motions by learning shared dynamics in a differentiable world model and improving its policy through imagined rollouts. On the 129,514-motion BONES-SEED training benchmark, it exceeds SONIC’s released-checkpoint normalized return after 3 hours on one H100, with 25 failures versus 179 under differing failure criteria; the comparison is not hardware- or data-matched.

Key insights

• A world model learned from simulator data can provide useful policy gradients without differentiating through the simulator itself.
• The learned model’s short-horizon prediction error remains in the reported 0.007–0.011 range as the motion dataset grows from 1 to 264 hours.
• Both faster imagined rollouts and gradient-based updates contribute to efficiency: rollouts are over 10× faster than MuJoCo Warp at practical batch sizes, and the method uses about 64× fewer imagined steps than PPO on model rollouts to reach its best training return.

Summary by alphaXiv.