Reproduce “SciGame: Let Agents Play the Scientific Discovery Games in the Real and Counterfactual World” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yusheng Zhao, Bowei He, Xue Liu, Mingxiao Li
SciGame benchmarks whether agents can identify or discover predictive scientific relations in paired simulated worlds, including counterfactual laws. Agents use prior knowledge, preset observations, or adaptive experiments; evaluation separates hidden-input prediction, exact-expression recovery, and explanation scores. On Temporal Counterfactual Discovery, GPT-6 Astra’s strict success was 16.7% with Adaptive versus 9.7% with four preset experiments, but budgets differed.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.