Reproduce “OpenDecoder: Open Large Language Model Decoding to Incorporate Document Quality in RAG” on OpenResearch

About this paper

Fengran Mo, Zhan Su, Yuchen Hui, Jinghan Zhang, Jia Ao Sun, Zheyuan Liu, Chao Zhang, Tetsuya Sakai and 1 more

Researchers from multiple academic institutions developed OpenDecoder, a framework that integrates explicit document quality indicators directly into the large language model's decoding process for Retrieval-Augmented Generation. This method improved RAG system robustness against noisy retrieved information and consistently outperformed baseline models across various noise conditions.

Key insights

• Explicitly guiding the LLM's attention mechanism with external relevance signals significantly enhances RAG system robustness, enabling better tolerance for noisy and irrelevant retrieved documents.
• Modulating attention based on explicit indicators allows the LLM to dynamically adjust its reliance on retrieved context versus its parametric knowledge, moving beyond implicit internal judgments.
• Robustness training, including injecting noise and varying document order, effectively mitigates LLM positional biases and encourages the model to prioritize external relevance cues over mere document placement.

Summary by alphaXiv.