Reproduce “Towards a General Humanoid Loco-Manipulation Model via Egocentric Whole-Body Human Data Pretraining” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chongyang Xu, Zhao Wu, Jin Chen, Yiming Jiang, Jinhui Ye, Yuming Jiang, Shifeng Zhang, Ziliang Feng and 4 more
λ₀ uses synchronized whole-body human motion to teach humanoid robots to coordinate movement and object handling, then adapts the policy with robot demonstrations. Its HUMANVERSE-500 dataset contains 500 hours of paired egocentric video and body and hand motion. On four real-world tasks, it achieves 62.5% success and 80.4% progress; evaluation is limited to the Unitree G1.
Key insights
- Synchronized body and hand motion offers supervision for the locomotion, posture, and interaction coordination needed in loco-manipulation, complementing rather than replacing robot demonstrations.
- The whole-body data stage has a larger observed effect on real-world performance than the initial hand-focused pretraining stage; using more HUMANVERSE-500 data is also associated with lower validation loss and higher task progress.
- Human demonstrations can help with objects not seen in robot training, although performance on these objects remains below performance on robot-seen objects.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.