Reproduce “Privacy-Aware Decoding: Mitigating Privacy Leakage of Large Language Models in Retrieval-Augmented Generation” on OpenResearch

About this paper

Haoran Wang, Xiongxiao Xu, Baixiang Huang, Kai Shu

Privacy-Aware Decoding (PAD) mitigates privacy leakage in Retrieval-Augmented Generation (RAG) by adaptively injecting calibrated Gaussian noise into Large Language Model (LLM) token logits during inference. This method significantly reduces verbatim data extraction from retrieved documents while maintaining or improving generation utility across various sensitive datasets and LLMs.

Key insights

• The decoding phase of RAG systems represents a critical, often overlooked, leakage channel where LLMs can directly emit sensitive content from retrieved documents.
• Adaptive noise injection, which is sensitive to both the model's confidence and the linguistic context, can effectively balance privacy protection with output utility.
• Quantifiable privacy guarantees, such as those derived from Rényi Differential Privacy (RDP), can be achieved for adaptively protected tokens during inference, enabling principled privacy-utility trade-offs.

Summary by alphaXiv.