Reproduce “E3: Issue-Level Backtesting for Automated Research Critique” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yashwardhan Chaudhuri, Sanyam Jain, Paridhi Mundra
E3 helps reviewers identify paper concerns by locating each one, explaining its relevance, and suggesting evidence or analysis to resolve it. The authors tested E3 on 100 ICLR 2026 papers against human reviews and two language-model flaw-finders, using a model-judged union of 4,598 issues. E3’s partial-inclusive recall was 90.2%, versus 74.7% for GPT; this measures coverage only within that union.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.