Reproduce “Using Vision-Language Models as Proxies for Social Intelligence in Human-Robot Interaction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Fanjun Bu, Melina Tsai, Audrey Tjokro, Tapomayukh Bhattacharjee, Jorge Ortiz, Wendy Ju
The authors propose a two-stage system for a café robot to decide whether to approach, leave, or probe. A lightweight detector flags gaze shifts or entry into personal space, prompting a video-language model to interpret recorded video and recommend an action. In 30 held-out episodes, it identified preamble timing for 43 interactants, versus 18 with distance-only triggering; this replay test does not establish live autonomy or general social intelligence.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.