Reproduce “Mitigating Bias in RAG: Controlling the Embedder” on OpenResearch

About this paper

Taeyoun Kim, Jacob Springer, Aditi Raghunathan, Maarten Sap

Carnegie Mellon University researchers investigated how biases from Large Language Models, embedders, and corpora interact within Retrieval-Augmented Generation (RAG) systems. They found that strategically reverse-biasing the embedder can effectively neutralize overall RAG system bias, especially for gender-related biases, without compromising retrieval utility.

Key insights

• The overall bias of a RAG system is a result of "bias conflict" among its components, demonstrating that a simple summation of individual component biases does not predict the total system bias.
• Achieving a debiased RAG system frequently necessitates reverse-biasing the embedder, strategically shifting its bias to compensate for the LLM's inherent biases.
• Different LLMs exhibit varying sensitivity to embedder bias, and this sensitivity can differ between types of bias (e.g., gender vs. political), highlighting the need for context-specific mitigation.

Summary by alphaXiv.