Reproduce “Avoiding Knowledge Edit Skipping in Multi-hop Question Answering with Guided Decomposition” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yi Liu, Xiangrong Zhu, Xiangyu Liu, Wei Wei, Wei Hu
Researchers from Nanjing University developed IRAKE, an iterative retrieval-augmented framework that mitigates "edit skipping" in multi-hop question answering for large language models by intelligently guiding question decomposition. This method achieved up to 55.24% multi-hop accuracy on the MQuAKE-2002 dataset with LLaMa-3-8B, surpassing previous approaches by effectively incorporating edited knowledge.
Key insights
- A granularity mismatch between LLM-generated subquestions and the target edited facts is a critical bottleneck in multi-hop knowledge editing.
- Using atomic questions derived from edited facts provides more effective guidance for decomposition than directly using factual answers, preventing confusion from counterfactual information.
- Leveraging both individual fact guidance and patterns from successfully solved multi-hop cases, combined with a backtracking mechanism, significantly improves the recall and integration of edited knowledge.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.