Reproduce “DIVERGE: Diversity-Enhanced RAG for Open-Ended Information Seeking” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tianyi Hu, Niket Tandon, Akhil Arora
Researchers from Aarhus University and Microsoft Research developed DIVERGE, an iterative agentic RAG framework that explicitly models viewpoints to generate diverse, high-quality responses for open-ended questions. DIVERGE achieved up to a 2-fold improvement in semantic and coverage diversity with minimal quality degradation across various large language models and real-world datasets.
Key insights
- Standard RAG systems, while effective for factual correctness, often reduce output diversity compared to non-retrieval LLMs, and simple retrieval-side diversity techniques do not effectively counter this homogenization.
- Explicitly modeling diverse viewpoints during both the retrieval and generation phases is crucial for overcoming the inherent homogenization tendencies of LLMs in open-ended information-seeking tasks.
- Novel evaluation metrics, such as Coverage Diversity and a Unified Diversity-Quality Harmonic Score, are essential for accurately assessing the diversity-quality trade-off in open-ended question answering scenarios.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.