Reproduce “NVP-HRI: Zero Shot Natural Voice and Posture-based Human-Robot Interaction via Large Language Model” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuzhi Lai, Shenghai Yuan, Youssef Nassar, Mingyu Fan, Thomas Weber, Matthias Rätsch
A system named NVP-HRI integrates natural voice commands, deictic (pointing) posture, and a Large Language Model with zero-shot semantic segmentation to enable robots to interact with unknown objects without prior training. This approach enhanced interaction efficiency by up to 65.2% compared to existing methods and improved collision-free trajectory success rates from 73% to 94%.
Key insights
- Integrating zero-shot semantic segmentation with natural voice and deictic posture allows robots to robustly understand and interact with previously unknown objects without requiring prior training.
- Employing a cross-check mechanism for LLM-generated action sequences significantly reduces the risk of hallucination and improves the reliability of collision-free robot trajectories.
- A multi-modal HRI system leveraging natural communication methods offers substantial efficiency gains and maintains consistent performance across diverse user age groups.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.