Reproduce “VeriGraph: Towards Verifiable Data-Analytic Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiajie Jin, Zhao Yang, Wenle Liao, Yuyang Hu, Guanting Dong, Xiaoxi Li, Yutao Zhu, Zhicheng Dou
Researchers at Renmin University of China developed VeriGraph, a neuro-symbolic framework that enables LLM-based agents to construct an explicit, verifiable evidence graph during data-intensive analytical tasks. This framework achieves an 87.61% grounding rate for claim verifiability, making agent outputs transparent and auditable, while maintaining competitive overall performance against larger models.
Key insights
- Data-intensive analytical reasoning is best captured as a heterogeneous Directed Acyclic Graph (DAG), unifying deterministic code execution and natural language semantic derivations, rather than a linear sequence.
- Explicitly embedding graph construction primitives within the agent's action space and training for their use significantly enhances output verifiability, surpassing simple tool use or structured prompting.
- Training agents to build verifiable evidence graphs requires a multi-faceted reward system that evaluates computational integrity and derivational validity, in addition to final answer correctness.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.