Reproduce “RECAP: A Resource-Efficient Method for Adversarial Prompting in Large Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Rishit Chugh

RECAP is a method for generating adversarial prompts in Large Language Models that significantly reduces computational requirements. This approach achieves competitive attack success rates while operating substantially faster than existing methods and uniquely enabling testing of black-box LLMs, which are inaccessible to gradient-based attacks.

Key insights

  • Decoupling adversarial prompt generation from real-time gradient computations through a RAG-inspired framework drastically improves computational efficiency, making LLM red teaming more accessible.
  • A pre-compiled database of categorized, effective adversarial examples, coupled with a hierarchical retrieval strategy based on prompt intent, can achieve competitive attack success rates without extensive training.
  • The RAG-based approach enables adversarial testing of black-box LLMs, a capability largely absent in traditional gradient-based methods due to their reliance on internal model access.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.