Reproduce “Profiling a Frontier Agent with EBench: From Direct Control to Harness with Policy” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yifei Yao, Yude Zou, Guanbo Jiao, Xing Gao, Ning Gao, Yukai Wang, Jiantong Chen, Zihou Zhu and 7 more
GPT-6-Astra directly controls robot manipulation in EBench and can also call post-trained policies to execute motion. The benchmark covers 510 simulated episodes across 26 tasks, using task success and partial stage completion. Direct control reaches 46.7% success; the InternW0-∆ harness reaches 56.0%, above the strongest standalone policy, but high-precision performance remains weak and handoffs can regress.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.