Reproduce “GRACE: Graph-Grounded Reflective Agent Copilot Engine for Expert-in-the-Loop Knowledge Expansion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
John Seon Keun Yi, Joshua R. Minot, Dokyun Lee
GRACE is a claim-level retrieval and knowledge-expansion framework that connects LLM-generated statements to trusted source claims in a weighted graph. It separates grounded, refuted, and unresolved claims, directs expert attention toward high-value uncertainties, and raises QASPER accuracy from 14.13% with abstract-only context to 49.13% after three verification rounds while reducing the unresolved claim frontier.
Key insights
- External knowledge anchors are needed alongside model self-consistency: repeated generations receive strong support only when connected to trusted priors, reducing the risk of endorsing shared hallucinations.
- The Boundary category preserves potentially useful new or disputed information instead of automatically treating every unsupported claim as false.
- Compact, verified claim graphs can improve retrieval and reduce review demands over time, although classifier conservatism and verifier abstention remain important limitations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.