Reproduce “Ramanujan Challenge Results: AI-Assisted Evaluation Pipeline for Multi-Format Proofs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Alan Sorani, Ashvni Narayanan, Michael Shalyt, Tali Monderer, Rotem Kalisch, Hila Barkan, Elyasheev Leibtag, Shachar Weinbaum and 3 more
The authors present a decision-support pipeline for reviewing submissions that combine written proofs and code. It summarizes prose and code separately, compares them for conflicts, and supports human review. Against one-shot Claude Opus 5 judging, false-positive “Correct” labels fell from 23.4% to 5.8% for proofs and 38.3% to 3.3% for code. The pipeline does not replace expert review.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.