Reproduce “HARMONI: Multimodal Personalization of Multi-User Human-Robot Interactions with LLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jeanne Malécot, Hamed Rahimi, Jeanne Cattoni, Marie Samson, Mouad Abrini, Mahdi Khoramshahi, Maribel Pino, Mohamed Chetouani
HARMONI links multimodal speaker perception, short-term conversational context, speaker-specific long-term profiles and memories, and context-conditioned response generation for multi-user dialogue. On a custom set of 45 videos involving six users, it achieves 89.60% accuracy in speaker recognition and 97.90% in profile retrieval. These results measure components, not sustained effectiveness in real-world care.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.