Reproduce “HATS: A Human-Agent Teleoperation System for Multi-Arm Data Collection” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zesen Lin, Jian-Jian Jiang, Haoming Cen, Xiao-Ming Wu, Dandan Zhang, Wei-Shi Zheng
HATS introduces a human-agent teleoperation system that enables a single operator to efficiently collect high-quality data for multi-arm tasks by decoupling control and employing MLLMs for assistive arms. The system achieved an 86.4% average task success rate, comparable to dual-human expert teams, and generated demonstration trajectories that produced robust downstream manipulation policies.
Key insights
- Decoupling human and agent control within a multi-arm teleoperation system effectively reduces operator cognitive load and coordination costs while maintaining high task success rates.
- MLLMs can be safely and effectively integrated into real-time human-in-the-loop robotics for high-level planning and assistive control, provided robust validation and human supervision mechanisms are in place.
- Generating spatially consistent and fluid motion trajectories through a human-agent collaborative system is crucial for producing high-quality demonstration data, which leads to more stable and accurate imitation learning policies.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.