Reproduce “RAG Safety: Exploring Knowledge Poisoning Attacks to Retrieval-Augmented Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tianzhe Zhao, Jiaoyan Chen, Yanchi Ru, Haiping Zhu, Nan Hu, Jun Liu, Qika Lin
Researchers conducted the first systematic study of data poisoning attacks on Knowledge Graph-Retrieval-Augmented Generation (KG-RAG) systems. They demonstrated that minimal graph perturbations can lead to substantial performance degradation, such as an 81% relative drop in Exact Match for RoG on WebQSP, primarily by compromising the retrieval stage and showing that stronger KG reasoning in LLMs correlates with higher susceptibility to poisoned data.
Key insights
- The retrieval component of KG-RAG systems is the most critical vulnerability, consistently retrieving injected malicious triples.
- LLMs fine-tuned for strong KG reasoning exhibited higher susceptibility to poisoned knowledge, suggesting a trade-off between performance and robustness.
- Even minor, stealthy perturbations to the Knowledge Graph can cause substantial performance degradation in KG-RAG systems, indicating high sensitivity to data poisoning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.