Reproduce “APRES: An Agentic Paper Revision and Evaluation System” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Bingchen Zhao, Jenny Zhang, Chenxi Whitehouse, Minqi Jiang, Michael Shvartsman, Abhishek Charnalia, Despoina Magka, Tatiana Shavrina and 3 more
Researchers from Meta Superintelligence Labs and the University of Edinburgh developed APRES, an agentic system that first identifies evaluation rubrics highly predictive of a paper's future citations and then uses these rubrics to iteratively revise manuscripts. This framework improved citation prediction by 19.6% Mean Absolute Error compared to baseline methods, with human experts preferring the APRES-revised papers 79% of the time over originals.
Key insights
- Large language models can effectively discover evaluation rubrics that are more predictive of a paper's future citation impact than human-assigned scores or standard paper embeddings.
- Iterative, agentic paper revision guided by these impact-predictive rubrics consistently enhances paper quality, particularly benefiting manuscripts on the borderline of acceptance.
- LLM-based review committees exhibit significantly higher consistency in evaluations compared to human peer review committees, suggesting a more stable assessment process.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.