Reproduce “STEM: Structure-Tracing Evidence Mining for Knowledge Graphs-Driven Retrieval-Augmented Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Peng Yu, En Xu, Bin Chen, Haibiao Chen, Yinfei Xu
STEM addresses multi-hop knowledge-graph question answering by translating questions into linked relational assertions, grounding them as graph triples, and using Triple-GNN guidance to retrieve connected evidence. On CWQ with GPT-4o, it reports 74.09% Hit@1, versus 69.61% for RoG. Evaluations use Freebase and fine-tuned modules; the authors do not present STEM as a general-purpose zero-shot method.
Key insights
- Aligning a question’s relational structure with the KG schema before retrieval helps address mismatches between ordinary phrasing and graph-specific relation vocabulary.
- Global structural guidance complements local semantic similarity; ablations show that removing structural biases lowers both answer scores and reasoning-path coverage, with triple-level bias especially useful.
- Precision and Breadth make answer coverage an explicit retrieval trade-off: broader branching can capture more answers but increases retrieval time.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.