Reproduce “Automatic Paper Reviewing with Heterogeneous Graph Reasoning over LLM-Simulated Reviewer-Author Debates” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shuaimin Li, Liyang Fan, Yufang Lin, Zeyang Li, Xian Wei, Shiwen Ni, Hamid Alinejad-Rokny, Min Yang
ReViewGraph predicts accept/reject outcomes from full papers by simulating reviewer-author exchanges and encoding opinions and their relations in a typed graph for a trained graph classifier. On the ICLR 2025 test set, it achieves Macro F1 71.65, versus 59.26 for CycleReviewer-70B. This is a benchmark result, not evidence of improved real-world peer review.
Key insights
- Explicitly modeling multi-round reviewer-author debates and their argumentative relations within a structured graph is essential for capturing nuanced dynamics and improving the accuracy of automated review decisions.
- The heterogeneous nature of the debate graph, with distinct node and edge types, allows for more semantically expressive representations of review interactions, which enhances decision-making and interpretability.
- Superior performance and generalizability can be achieved in automated peer review by leveraging structured graph reasoning over LLM-generated debate content, effectively bypassing the need for intensive LLM parameter updates.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.