Reproduce “Graphs Don't Stay Secret: Practical Subgraph Reconstruction Attacks on Defended Graph RAG” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Minkyoo Song, Jaehan Kim, Myungchul Kang, Hanna Kim, Seungwon Shin, Sooel Son
Researchers from KAIST developed GRASP, a targeted subgraph reconstruction attack that successfully extracts one-hop relational profiles from Graph Retrieval-Augmented Generation (RAG) systems, even when protected by prompt-based safeguards. The attack achieved RType F1 scores up to 82.9% on real-world knowledge graphs by reframing extraction as a legitimate context-processing task, using instance-grounded outputs, and employing adaptive query strategies.
Key insights
- Traditional prompt-based RAG defenses, summarization techniques, and even advanced decoding-stage mitigations are largely insufficient against task-reframed, instance-grounded attacks like GRASP, which can bypass explicit intent detection.
- Enforcing a fixed, line-based output format with per-instance identifiers is crucial for accurate and non-hallucinated relation extraction, substantially improving precision by delimiting individual relation records.
- Adaptive querying with diverse templates and a dynamic prompt scheduler is essential for effectively reconstructing subgraphs by mitigating saturation and efficiently expanding coverage of new relations.
- The instruction-following capabilities of more capable LLMs can inadvertently amplify adversarial risks, as these models more faithfully execute the constrained extraction task posed by GRASP.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.