Reproduce “Ego2Contact: Predicting Touch from Egocentric Human Video for Contact-Rich Robot Manipulation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Qianqian Yang, Dayou Li, Bangya Liu, Zihao Zhu, Nuo Chen, Siyuan Xu, Hezhen Hu, Chen Yu and 6 more

Ego2Contact learns a visual-to-touch predictor from paired human video and glove measurements, then adapts it to robot hand regions. In simulation, predicted touch raises Bottle Cap Turning success from 64.1% to 71.0%, compared with 74.1% using ground-truth touch. It improves three of four tasks, but barely changes Pen Cap Removal; real-robot rollouts still use glove measurements.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.