Reproduce “When Embedding-Based Defenses Fail: Rethinking Safety in LLM-Based Multi-Agent Systems” on OpenResearch

About this paper

Lingxi Zhang, Guangtao Zheng, Hanjie Chen

Researchers identified a fundamental vulnerability in embedding-based multi-agent system defenses, demonstrating that these methods can be circumvented by 'near-benign' adversarial messages crafted to evade detection. They introduced a confidence-guided defense utilizing token-level model confidence, which proved more robust across various LLM backbones and tasks, recovering significant performance compared to existing baselines.

Key insights

• Embedding-based defenses are fundamentally vulnerable to attacks that craft messages whose embeddings lie within the learned 'benign acceptance region,' thus evading outlier detection.
• Internal model signals, specifically token-level confidence, offer a complementary and more robust defense mechanism against sophisticated, embedding-obfuscating attacks.
• The effectiveness of defensive signals and the spread of malicious content are heavily influenced by communication topology and degrade over multiple communication rounds, highlighting the importance of early intervention.

Summary by alphaXiv.