Reproduce “Exposing Weak Links in Multi-Agent Systems under Adversarial Prompting” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Nirmit Arora, Sathvik Joel, Ishan Kavathekar, Palak, Rohan Gandhi, Yash Pandya, Tanuja Ganu, Aditya Kanade and 1 more

Researchers from Microsoft Research, IIIT Hyderabad, and EPFL introduced SAFEAGENTS and DHARMA, a unified framework and a diagnostic metric, respectively, to expose security vulnerabilities in Multi-Agent Systems (MAS) under adversarial prompting. Their analysis revealed that centralized MAS often exhibit higher attack success rates (e.g., up to 92.52% ARIA 4 in ASB for Magentic) than single-agent systems, and identified specific weak links related to planning, context fragmentation, and execution across various MAS architectures and safety benchmarks.

Key insights

  • Centralized MAS are not inherently safer; they can amplify Attack Success Rates (ARIA 4), with centralized Magentic showing up to 83.70% ARIA 4 in RedCode compared to 62.5% for SA, by efficiently executing harmful goals when alignment fails.
  • Granular design choices, such as sub-agent autonomy, planning strategies (stratified vs. combined), and context organization, are more critical determinants of MAS security than high-level architectural patterns.
  • Specific weak links in MAS include inadequate fallback mechanisms, context fragmentation that prevents sub-agents from recognizing overall malicious intent, and 'blind' execution of pre-generated harmful plans without continuous safety reassessment.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.