Reproduce “Mixed-Initiative Dialog for Human-Robot Collaborative Manipulation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Albert Yu, Chengshu Li, Luca Macesanu, Arnav Balaji, Ruchira Ray, Raymond Mooney, Roberto Martín-Martín

MICoBot, a system developed by UT Austin and collaborators, integrates mixed-initiative natural language dialog with mixed-initiative human-robot physical interaction to enable dynamic task allocation for collaborative manipulation. In real-world human-robot collaboration, it achieved a 77.8% task success rate, a substantial improvement over a pure LLM baseline, and was preferred by 77.8% of human participants.

Key insights

  • Unifying mixed-initiative natural language dialog with mixed-initiative physical interaction significantly enhances human-robot collaboration, enabling flexible adaptation to varying human behavior and preferences.
  • Dynamically estimating human willingness to assist through dialog analysis (`p_H,t`) is critical for optimal task allocation, allowing the robot to effectively leverage human input and manage its own limitations.
  • A hierarchical planning architecture, combining LLM-driven strategy generation with constrained optimization, enables robust negotiation, adaptive task execution, and improved user experience in complex collaborative tasks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.