Reproduce “SEER: Self-Aligned Evidence Extraction for Retrieval-Augmented Generation” on OpenResearch

About this paper

Xinping Zhao, Dongfang Li, Yan Zhong, Boren Hu, Yibin Chen, Baotian Hu, Min Zhang

SEER, developed by researchers from Harbin Institute of Technology (Shenzhen), Peking University, Zhejiang University, and Huawei Cloud, introduces a self-aligned evidence extraction framework for Retrieval-Augmented Generation (RAG). The system improves question answering accuracy by up to 13.5% while reducing the average evidence input length to Large Language Models by 9.25 times.

Key insights

• A model-based, self-aligned approach can effectively and efficiently extract high-quality, concise, and context-faithful evidence, outperforming heuristic-based methods.
• Optimizing evidence for a combination of faithfulness, helpfulness, and conciseness collectively leads to superior RAG performance and significant computational savings.
• Incorporating listwise ranking signals through Lambda Preference Optimization (LPO) is crucial for achieving better alignment and more nuanced preference learning compared to pairwise optimization methods.

Summary by alphaXiv.