Reproduce “Learning to Plan for Retrieval-Augmented Large Language Models from Knowledge Graphs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Junjie Wang, Mingyang Chen, Binbin Hu, Dan Yang, Ziqi Liu, Yue Shen, Peng Wei, Zhiqiang Zhang and 5 more
Wang et al. introduced the Learning to Plan from Knowledge Graphs (LPKG) framework, which utilizes Knowledge Graphs to generate structured and accurate planning data for fine-tuning large language models. This approach enhanced the planning capabilities of smaller LLMs for complex retrieval-augmented question answering, demonstrating superior performance compared to distillation methods and achieving over 20% average improvement in Precision and Recall on the new CLQA-Wiki benchmark compared to in-context learning baselines.
Key insights
- Knowledge Graphs offer a scalable and reliable source for constructing structured, accurate planning data, providing a more robust alternative to manual annotation or potentially fallible knowledge distillation from teacher models.
- Fine-tuning even smaller LLMs with KG-derived planning data substantially improves their ability to decompose complex questions and formulate effective multi-step reasoning plans, allowing them to achieve competitive performance against larger models.
- Decoupling the planning LLM from the QA LLM within a retrieval-augmented generation (RAG) system enables specialized optimization for each component, leading to more robust and accurate responses in complex QA tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.