Reproduce “Breaking the 15% Barrier: A Real-World Data-Driven System for Proactive Social Robot Triggered by User Nonverbal Cues” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuga Yano, Yuki Okafuji, Ryo Miyoshi, Sanae Yamashita, Yoshiki Ohira
Researchers from Kyushu Institute of Technology and CyberAgent developed a data-driven system that enables social robots to proactively respond to user nonverbal cues in real time, moving beyond traditional speech-only interactions. Their analysis of real-world retail interactions found that 15.3% of robot utterances were nonverbal-triggered, with their system achieving a micro-average F1-score of 0.8 for nonverbal cue recognition and 49.1% intent agreement for LLM-generated responses.
Key insights
- A substantial 15.3% of robot utterances in a real-world retail environment were found to be triggered by user nonverbal behaviors, demonstrating the critical need to move beyond speech-only interaction paradigms.
- Real-time recognition of multiple nonverbal cues is achievable with efficient model architectures, overcoming the high latency challenges of advanced Vision-Language Models for practical HRI deployment.
- Nonverbal cues predominantly elicit social and engagement-invitation robot responses (44.0% "Social & Greeting", 29.9% "Engagement invitation"), highlighting their role in initiating natural and proactive human-robot interactions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.