Reproduce “Guardians of the Agentic System: Preventing Many Shots Jailbreak with Agentic System” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Saikat Barua, Mostafizur Rahman, Md Jafor Sadek, Rafiul Islam, Shehenaz Khaled, Ahmedul Kabir

This research evaluates the security of LLM-based autonomous agents against sophisticated attacks such as many-shot jailbreaking and deceptive alignment. It introduces dynamic, multi-agent simulation frameworks to assess agent vulnerabilities and the efficacy of defensive measures, finding that GEMINI 1.5 pro achieved 94% accuracy in detecting malicious agents in a Reverse Turing Test and that longer prompts consistently increased jailbreaking success rates.

Key insights

  • Static safety mechanisms are inadequate against advanced, persistent adversarial attacks like many-shot jailbreaking and deceptive alignment, requiring more dynamic and adaptive defenses.
  • The length and specific characteristics of jailbreak prompts significantly influence attack success rates, highlighting varying vulnerabilities across different LLM models.
  • Effective AI safety requires a holistic approach that integrates active monitoring, adaptive intervention, and multi-agent interaction analysis beyond isolated, single-turn assessments.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.