Reproduce “Cross-Embodiment Dexterous Grasping with Reinforcement Learning” on OpenResearch

About this paper

Haoqi Yuan, Bohan Zhou, Yuhui Fu, Zongqing Lu

CrossDex lets one policy control different robotic hands through a shared human-like pose interface, hand-specific retargeting, and fingertip and palm observations. The authors train with four hands and test on two unseen hands. In simulation, its vision policy reaches a 0.352 success rate on unseen hands, versus 0.210 for a baseline; evidence is limited to the tested hands and grasping task.

Key insights

• Human hand eigengrasps provide an effective, low-dimensional, and universal action representation for controlling diverse robotic dexterous hands.
• Restricting observations to general geometric features (fingertip/palm positions) and removing embodiment-specific proprioception is critical for achieving zero-shot generalization to unseen hand morphologies.
• Co-training a single policy across multiple distinct dexterous hands within a parallel simulation environment encourages the acquisition of more generalizable and transferable grasping skills.

Summary by alphaXiv.