Reproduce “KAG-Thinker: Interactive Thinking and Deep Reasoning in LLMs via Knowledge-Augmented Generation” on OpenResearch

About this paper

Dalong Zhang, Jun Xu, Jun Zhou, Lei Liang, Lin Yuan, Ling Zhong, Mengshu Sun, Peilong Zhao and 11 more

KAG-Thinker, developed by Knowledge Engine Team @Inclusion AI, Ant Group, proposes a framework that enhances Large Language Models' reasoning for complex multi-hop Q&A by simulating human-like "slow thinking" via logical forms and intelligent knowledge management. It achieves new state-of-the-art results on general domain benchmarks, outperforming prior RAG and reinforcement learning methods by an average of 4.1% EM score, and demonstrates improved accuracy and stability in medical Q&A applications.

Key insights

• Decomposing complex multi-hop problems into logical forms enables a more rigorous, stable, and interpretable reasoning process for LLMs.
• Intelligent arbitration between an LLM's internal knowledge and external retrieval, guided by confidence assessment, reduces noise and improves efficiency.
• Supervised fine-tuning with a structured, multi-turn reasoning corpus can achieve state-of-the-art deep reasoning performance without relying on reinforcement learning.

Summary by alphaXiv.