Reproduce “OASIS: From Simulation Data Collection to Real-World Humanoid Loco-Manipulation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zehao Yu, Jiakun Zheng, Weiji Xie, Jiyuan Shi, Chenyun Zhang, Chenjia Bai, Xuelong Li

OASIS collects humanoid demonstrations in simulation, then replays recorded motions offline with varied textures, lighting, and camera views. It uses these visual variants to train a controller for real-world deployment. On four Unitree G1 tasks, simulation-only training was comparable to real-only training with 50-trajectory budgets, but scored higher on two tasks, tied on one, and lower on one.

Key insights

  • Decoupling real-time VR teleoperation for trajectory collection from computationally intensive, high-fidelity rendering allows for efficient generation of core kinematic data which can then be massively augmented offline.
  • Extensive visual domain randomization in simulation, particularly lighting randomization, is critical for bridging the sim-to-real gap and enabling effective zero-shot transfer to real humanoid robots.
  • Training the high-level planner with reference motion commands as proprioception and incorporating a curriculum-based rollout mechanism improves policy stability and generalization by mitigating compounding errors over long-horizon autoregressive deployment.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.