Reproduce “ExplorationBench: Measuring AI Systems' Exploration in Verifiable Alien Worlds” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ming Zhang, Zhenghao Xiang, Peizhong Gao, Yujiong Shen, Yuhui Wang, Zhonghan Yue, Shihan Dou, Zhangyue Yin and 12 more
ExplorationBench tests whether systems can learn unfamiliar rules by choosing experiments in two deterministic synthetic worlds. Systems receive misleading manuals, probe the worlds, then answer held-out tasks graded by interpreters or proof-checkers. The strongest AlienCode trajectory reaches 89.0% accuracy after four exploration rounds, compared with 0.5–11.0% without feedback; this demonstrates learning in these worlds, not real-world scientific discovery.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.