Reproduce “RouterKGQA: Specialized--General Model Routing for Constraint-Aware Knowledge Graph Question Answering” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Bo Yuan, Hexuan Deng, Xuebo Liu, Min Zhang
Researchers from Harbin Institute of Technology developed RouterKGQA, a framework for Knowledge Graph Question Answering that routes queries between a specialized language model and a general language model for efficient, accurate responses. It improves precision through constraint-aware reasoning paths and cost-effectiveness via an optimized repair mechanism, achieving F1 score increases of 4.30 points on WebQSP and 2.84 points on CWQ, with an average of 1.15 LLM calls per question.
Key insights
- Dynamically routing queries between an efficient specialized model and a powerful general repair model effectively balances high accuracy with reduced computational costs, leveraging each model's strengths.
- Explicitly encoding and utilizing Constraint-aware Reasoning Paths (CRPs) in the specialized model significantly improves precision by capturing implicit question constraints and filtering irrelevant answers.
- The task of correcting unreachable reasoning paths can be made cost-effective by employing an optimized general agent workflow that utilizes guided beam search over relations and semantic filtering, rather than brute-force entity-relation search.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.