Reproduce “SHAPE: Discrete Supervision and Parallel Continuous Streams for Learning Dexterous Manipulation from Egocentric Human Data” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yaping Li, Zhihong Liu, Jiacheng Li, Yi Yang, Wenchao Yue, Mingzhe Ye, Xiangcheng Liu, Yiyang Chen and 5 more
SHAPE trains a dexterous robot policy from human hand motion in end-effector space and robot demonstrations in end-effector and joint spaces. FAST action tokens supervise shared model components, while continuous streams provide executable actions. After human-only task fine-tuning, the end-effector stream succeeds in 21 of 60 trials, without task-specific robot demonstrations; pretraining included robot teleoperation, and human motions were retargeted.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.