Reproduce “Is My Data in Your Retrieval Database? Membership Inference Attacks Against Retrieval Augmented Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Maya Anderson, Guy Amit, Abigail Goldsteen
RAG-MIA tests whether a passage appears in a retrieval-augmented generation database by embedding the passage in a question that steers retrieval and asks for confirmation. Its black-box attack uses the answer, while gray-box analysis uses token log-probabilities. On HealthCareMagic with Flan and prompt #2, black-box TPR was 1.00 at 0.74 FPR; gray-box TPR was 0.83 at 0 FPR, with 0.99 AUC-ROC.
Key insights
- The RAG system's inherent "context grounding" capability, designed to base responses on retrieved information, paradoxically creates a vulnerability for membership inference.
- Carefully engineered attack prompts are essential for successfully activating retrieval and compelling the RAG's LLM component to disclose document membership.
- Initial defense strategies, which instruct LLMs to avoid directly confirming context content, show model-dependent effectiveness, highlighting varying compliance levels among generative models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.