Reproduce “RoboValue: A Fine-Grained Sim-and-Real Benchmark for Unified Evaluation of Robotic Value Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shengbang Liu, Zhengye Du, Zhilong Wan, Honghao Su, Chenxiang Xia, Chang Ge, Jinyang Xiao, Nan Wang and 11 more
RoboValue benchmarks whether robotic value models’ scores reflect task instructions and execution history, rather than only progress or success. It tests task understanding, progress, recovery, and consistency across valid subtask orders in simulation and real-world tasks. In tea-set examples, CSVC was −40.98 for Robo-Dopamine 2.0-8B Preview and 2.30 for RoboMeter; these different evaluation tracks are not a head-to-head comparison.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.