Reproduce “BEHAVIOR-Spatial Benchmark: Evaluating Spatial Understanding for Robotics” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sirui Chen, Huang Huang, Qineng Wang, Wenlong Huang, Manling Li, Ruohan Zhang, Li Fei-Fei, Jiajun Wu
BEHAVIOR-Spatial evaluates spatial question answering, point grounding, and robot action across shared task contexts. Its 3,256 questions cover 49 activities, with 1,240 grounding queries and manipulation and navigation tasks. In a 700-episode simulator prompt probe, oracle dot cues raise soft success from 16.1% to 38.0% and strict success from 10.0% to 24.9%, testing supplied targets, not learned language grounding.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.