Reproduce “How Reliable are LLMs as Knowledge Bases? Re-thinking Facutality and Consistency” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Danna Zheng, Mirella Lapata, Jeff Z. Pan
Researchers at the University of Edinburgh propose a new framework and metrics to evaluate large language models (LLMs) as knowledge bases, moving beyond simple accuracy to assess factuality and consistency when handling both seen and unseen knowledge. Their evaluation of 26 LLMs reveals that larger models are more consistently wrong, and fine-tuning improves handling of unseen knowledge while degrading seen knowledge performance.
Key insights
- LLMs that exhibit higher consistency in *correct* responses also tend to show higher consistency when *wrong*, posing a risk of confidently spreading misinformation.
- Performance on seen knowledge does not reliably predict performance on unseen knowledge, although incorporating 'unsure shots' in in-context learning can help align these two aspects.
- Base LLMs often overestimate their knowledge for numerical and temporal unseen facts, being more prone to providing misleading wrong answers rather than admitting uncertainty.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.