Reproduce “Constrained Human-AI Cooperation: An Inclusive Embodied Social Intelligence Challenge” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Weihua Du, Qiushi Lyu, Jiaming Shan, Zhenting Qi, Hongxin Zhang, Sunli Chen, Andi Peng, Tianmin Shu and 3 more
CHAIC is a simulated benchmark for testing whether embodied agents can assist partners with hidden goals and physical constraints, using egocentric RGB-D observations in long-horizon indoor and outdoor tasks. An LLM-based helper that combines object perception, partner-behavior recognition, memory, navigation, and planning achieves the best performance in most tasks, but remains below an oracle system, especially in goal inference, spatial reasoning, and emergency response.
Key insights
- Recognizing individual partner actions does not directly yield reliable intent inference: an action model reaches 86.1% validation accuracy, while task-level goal inference ranges only from 0.25 to 0.74.
- Language-model planning is most useful when physical constraints directly prevent the partner from completing a task, but the agents still struggle to choose efficient routes, exploit containers, remove obstacles, or prioritize the partner over independent progress.
- Explicit behavior and emergency-handling mechanisms remain important complements to general-purpose vision-language planning, particularly when the partner can become occluded, distant, or unexpectedly separated.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.