Reproduce “DS@GT ARC at LongEval: Citation Integrity and Factual Grounding in Scientific QA” on OpenResearch

About this paper

Brandon Michaels, Brendon Johnson

This work describes DS@GT ARC's submission to the CLEF 2026 LongEval Task 4, demonstrating that corrective Retrieval-Augmented Generation (RAG) interventions can improve factual grounding and citation integrity in scientific question answering systems, while leading frontier models tend to prioritize fluency and answer relevance over strict contextual attribution. The CRAG+CiteFix pipeline achieved higher faithfulness scores (Global Faithfulness 0.784, Citation Faithfulness 0.758) compared to frontier models, which showed lower faithfulness (e.g., GPT-5.5 Global Faithfulness 0.559, Citation Faithfulness 0.417) despite high traditional NLG metrics.

Key insights

• Frontier models, despite achieving high scores on traditional NLG metrics and answer relevancy, often produce answers predominantly from their parametric memory rather than strictly synthesizing from provided context, leading to lower factual grounding and attribution integrity.
• Active interventions such as pre-generation context filtering (CRAG) and post-generation citation pruning (CiteFix) can enhance the 'structural honesty' and factual grounding of RAG systems, prioritizing integrity even if it means reduced fluency or increased abstention.
• There is a clear divergence between traditional NLG evaluation metrics and metrics focused on factual consistency and citation integrity, underscoring the need for targeted evaluation methods for trustworthy scientific RAG.

Summary by alphaXiv.