Reproduce “exUMI: Extensible Robot Teaching System with Action-aware Task-agnostic Tactile Representation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yue Xu, Litao Wei, Pengyu An, Qingyu Zhang, Yong-Lu Li
Researchers from Shanghai Jiao Tong University introduced exUMI, an extensible robot teaching system, and Tactile Predictive Pretraining (TPP), an algorithm for learning action-aware tactile representations. The system enables efficient collection of large-scale, contact-rich multimodal datasets and significantly improves robot manipulation performance on force-sensitive tasks, achieving up to 55% higher success rates compared to vision-only policies.
Key insights
- A co-design approach, integrating hardware improvements for robust data collection with novel algorithmic approaches for representation learning, is effective for advancing tactile robot learning.
- Modeling tactile perception as an action-conditioned temporal prediction problem allows the learning of rich, dynamic, and generalizable tactile representations that capture contact mechanics.
- Efficient collection of large-scale, contact-rich "human play data" significantly mitigates the issues of tactile data scarcity and sparsity for pretraining.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.