Reproduce “Similar Is Not Useful: Selecting In-Context Demonstrations by Closed-Loop Utility” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiachen Li, Zhigen Zhao, Shihao Li, Dongmei Chen, Long Chen, Zhuo Xu
The authors test whether selecting demonstrations by their measured effect on a frozen robot policy improves task success over visual similarity. They estimate each example’s utility from closed-loop rollouts, then compare fixed pools of five on held-out LIBERO tasks. Across 19 screened tasks, utility selection reaches 66.4% success versus 55.7% for similarity, on one checkpoint; physical-robot transfer remains untested.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.