Reproduce “Simple Baselines are Competitive with Code Evolution” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yonatan Gideoni, Sebastian Risi, Yarin Gal
The authors test whether simple code-search methods can match elaborate language-model pipelines for mathematical discovery and agent design. They compare independent random sampling and sequential sampling conditioned on successful programs, and examine prompt, formulation, and evaluation effects. Under $20 per math problem, the methods match or beat ShinkaEvolve on 4/9 and 6/9 problems, respectively; results are limited to tested settings.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.