Reproduce “RHINO: Learning Real-Time Humanoid-Human-Object Interaction from Human Demonstrations” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jingxiao Chen, Xinyao Li, Jiahang Cao, Zhengbang Zhu, Wentao Dong, Minghuan Liu, Ying Wen, Yong Yu and 2 more

Researchers at Shanghai Jiao Tong University developed RHINO, a hierarchical learning framework for real-time, interruptible humanoid-human-object interaction learned from human demonstrations. It achieves high mean Average Precision in intention prediction and robust manipulation, demonstrating enhanced adaptability and scalability compared to end-to-end models on a Unitree H1 humanoid robot.

Key insights

  • Including granular human hand details (3D position, nearest object, IOU) significantly improves the accuracy of real-time human intention prediction.
  • Training with in-skill interruption data is essential for enabling robots to safely and flexibly respond to unexpected human interference during manipulation tasks.
  • A hierarchical framework provides greater scalability and robustness in complex HRI tasks compared to end-to-end models, especially for varied skill sets and out-of-distribution scenarios.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.