Reproduce “LIBERO-Pro2.0: A Benchmark for Evaluating the Robustness of Embodied Foundation Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xueyang Zhou, Yangming Xu, Qianjiang Li, Guiyao Tie, Junhao Wu, Ziheng Huang, Guowen Zhang, Zhijie Yuan and 5 more
LIBERO-Pro2.0 tests whether robot policies adapt to changed conditions before and during tabletop tasks. It defines 22 static perturbations and 20 runtime interventions across six dimensions. Evaluating 17 policies, the authors report an average 26.5-percentage-point runtime performance drop, with individual drops reaching 32.5 points. Matched results vary, so timing alone is not established as the cause.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.