Reproduce “SelfImprove-Bench: Evaluating Generalization in Self-Improving Agents” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Bangrui Xu, Jie Yang, Sidi Chen, Xuanhe Zhou, Jiantao Qiu, Qianyu Yang, Zijie Wang, Fan Wu and 1 more

SelfImprove-Bench evaluates whether gains from recursive self-improvement (RSI) methods transfer beyond the experience that prompted refinement. It tests six games across five levels, from replay to harder tasks and different capability categories, using normalized HCI. Three methods show positive HCI on 16 of 18 method–game pairs at G1, but HyperAgents is positive in only five of eight tested G4 directions.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.