Reproduce “LIBERO-RIG: A Hardware-Setup Robustness Benchmark for Vision-Language-Action Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Andrea Yaoyun Cui, Hanyu Wang, Xinyi Wang, Bangzheng Li, Lin Zhang, Zhenggang Tang, Shyamal Buch, Dejia Xu and 1 more
LIBERO-RIG evaluates whether robot policies retain performance when setup changes but the task remains fixed. In simulation, it varies eleven rig properties across five severity levels, holding the task constant. Across eight models and four suites, pooled success falls from 0.932 nominally to 0.767 at level 3 and 0.509 at level 5; results cover one simulated embodiment and single-axis changes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.