Reproduce “Corpus-informed Retrieval Augmented Generation of Clarifying Questions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Antonios Minas Krasakis, Andrew Yates, Evangelos Kanoulas
Researchers at the University of Amsterdam developed Fusion-in-Decoder (FiD) models for corpus-informed clarifying question generation, demonstrating their effectiveness and efficiency in grounding questions in available documents. The study quantifies a significant mismatch between existing datasets' ground truth clarifications and retrievable information, highlighting challenges for faithful generation.
Key insights
- Current clarifying question datasets, like MIMICS, exhibit poor alignment between ground truth facets and information truly present or retrievable in standard corpora.
- Training models with evidence explicitly aligned with ground truth facets significantly increases their faithfulness, prompting them to derive clarifications directly from the corpus.
- Existing dataset ground truths are often non-exhaustive and can contain a "taxonomy bias," potentially penalizing valid corpus-informed clarifications that diverge from predefined answers.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.