Reproduce “Context-aware Decoding Reduces Hallucination in Query-focused Summarization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhichao Xu
Context-aware Decoding (CAD) improves the factual consistency of Large Language Model (LLM) generated summaries, particularly in query-focused settings, by modifying the token generation process. This decoding strategy consistently reduced hallucination metrics across multiple models and datasets, although it incurred a twofold increase in computational overhead.
Key insights
- Context-aware Decoding (CAD) consistently reduces factual errors (hallucinations) in both news and query-focused summarization tasks across various LLMs.
- A notable trade-off exists in Query-Focused Summarization (QFS): increased factual consistency through CAD can lead to slightly lower ROUGE scores due to the abstractive nature of QFS references.
- The computational cost of CAD is approximately double that of vanilla decoding, directly impacting inference speed.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.