Reproduce “FormalEvolve: Neuro-Symbolic Evolutionary Search for Diverse Autoformalization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haijian Lu, Wei Wang, Jing Liu
FormalEvolve searches for a repertoire of Lean theorem statements rather than returning one translation, using compilation, semantic judging, patching, repair, and conservative AST rewrites to filter candidates before proving. Against a matched no-archive control, SH@100 rises from 0.530 to 0.580 on CombiBench and from 0.828 to 0.849 on ProofNet. Theorem-complete proofs also increase under a fixed budget, but not over every baseline.
Key insights
- Archive-based search improves coverage beyond stronger sampling-and-repair baselines by exploiting structural variation across previously generated Lean statements.
- Diversity is most useful when the benchmark involves domain or style shift: FormalEvolve’s gains were larger on CombiBench, while island structure and symbolic mutations had dataset-dependent effects.
- Coverage should include distributional measures such as Gini concentration and top-10-percent share, since many accepted statements can still be concentrated on a narrow set of problems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.