Reproduce “OmniUMI: Towards Physically Grounded Robot Learning via Human-Aligned Multimodal Interaction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shaqi Luo, Yuanyuan Li, Youhao Hu, Chenhao Yu, Chaoran Xu, Jiachen Zhang, Guocai Yao, Tiejun Huang and 2 more
OmniUMI provides a handheld, robot-free teaching system that synchronously records visual, motion, tactile, grasp-force, and external-wrench signals, then uses them in a diffusion policy with impedance-compatible execution. Bilateral feedback lets operators regulate contact during demonstrations. In nested-cup selective release, tactile input raised success from 20% to 100%; this comparison is task-specific, and human-alignment causality remains untested.
Key insights
- Integrating diverse physical sensors into a single, compact, and reusable robot-free interface is essential for overcoming the 'visuomotor bottleneck' in scalable robot learning for contact-rich tasks.
- Human-aligned bilateral feedback and embodied perception during data collection significantly improve the fidelity and human-likeness of recorded physical interaction signals.
- Maintaining hardware consistency between data collection and robot deployment, specifically through a shared gripper, is crucial for robust transfer of multimodal observations and learned policies.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.