Reproduce “HyperSim: A Holistic Sim-To-Real Framework For Robust Robotic Manipulation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Junyi Dong, Haotian Luo, Ziwei Xu, Shengwei Bian, Heng Zhang, Sitong Mao, Jingyi Guo, Yang Xu and 6 more

HyperSim, developed by CloudRobo Lab, Huawei, presents a holistic framework for robust sim-to-real transfer in robotic manipulation through high-fidelity environment synthesis, adversarial trajectory generation, and co-training with real data. The approach achieved up to a 95% overall success rate for a deep-bin transfer task on a humanoid robot using few-shot co-training and demonstrated enhanced robustness to dynamic perturbations.

Key insights

  • Integrating high-fidelity visual rendering via 3D Gaussian Splatting with adversarially diversified training data significantly reduces the visual and dynamic sim-to-real gap.
  • Adversarial trajectory generation, which includes dynamic perturbations and recovery actions, is crucial for fostering robust policies capable of handling unexpected real-world uncertainties.
  • Foundation models benefit substantially from high-quality synthetic data, achieving high zero-shot transfer performance and further improving with minimal real-world co-training.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.