Reproduce “GenH2R: Learning Generalizable Human-to-Robot Handover via Scalable Simulation, Demonstration, and Imitation” on OpenResearch

About this paper

Zifan Wang, Junyu Chen, Ziqing Chen, Pengwei Xie, Rui Chen, Li Yi

The GenH2R framework enables robots to learn robust human-to-robot (H2R) handover skills that generalize to novel objects and dynamic human behaviors by combining a scalable synthetic simulator, automated distillation-friendly demonstration generation, and forecast-aided 4D imitation learning. Experiments show it achieves a 70-90% real-world success rate in complex handover scenarios, outperforming baselines by up to 36.7% in challenging settings.

Key insights

• Large-scale synthetic data, generated through procedural methods and extensive 3D model repositories, is crucial for fostering robust generalizability in H2R handover policies, outperforming smaller real-world datasets.
• Balancing foresighted planning with real-time visual correlation through 'landmark planning' leads to higher quality and more distillable robot demonstrations for visuomotor learning, compared to purely foresighted or shortsighted approaches.
• Integrating spatio-temporal information (flow) with an auxiliary future prediction task within a 4D imitation learning framework enhances a robot's anticipatory capabilities and responsiveness to dynamic human handover motions.

Summary by alphaXiv.