Reproduce “The Silent Saboteur: Imperceptible Adversarial Attacks against Black-Box Retrieval-Augmented Generation Systems” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Hongru Song, Yu-an Liu, Ruqing Zhang, Jiafeng Guo, Jianming Lv, Maarten de Rijke, Xueqi Cheng

The authors introduce ReGENT, an attack that subtly changes a document so it enters a retrieval-augmented generation system’s shortlist and influences its answer. ReGENT trains a surrogate retriever and uses language-model feedback and reinforcement learning to choose word substitutions balancing retrieval, generation influence, and semantic preservation. In controlled tests, it reached 45% ASR on factual QA in a top-three-document setup.

Key insights

  • RAG systems are highly vulnerable to imperceptible attacks: subtle word substitutions (3-4% changes) in existing documents can hijack both retrieval and generation processes.
  • Vulnerabilities differ across QA scenarios, with stance-based QA showing higher retrieval attack success rates and factual QA demonstrating greater susceptibility in generation once context is manipulated.
  • The fidelity of the surrogate retrieval model, particularly through fine-grained training to capture hierarchical preferences, is critical for successfully executing black-box imperceptible attacks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.