Reproduce “Learn to Disguise: Avoid Refusal Responses in LLM's Defense via a Multi-agent Attacker-Disguiser Game” on OpenResearch

About this paper

Qianqiao Xu, Zhiliang Tian, Hongyan Wu, Zhen Huang, Yiping Song, Feng Liu, Dongsheng Li

Large Language Models can learn to provide secure responses to harmful queries while concealing their defensive intent, preventing attackers from refining jailbreaking methods. A multi-agent attacker-disguiser game framework achieves this by dynamically training LLMs to generate safe, relevant, and non-explicitly-refusing outputs, demonstrating a PoR of 89.83% on generated attacks and 94.46% on the XSAFETY benchmark.

Key insights

• Disguising defensive intent, rather than explicit refusal, can break the feedback loop that allows attackers to refine jailbreaking methods, leading to more robust LLM safety.
• Multi-agent adversarial training, combined with game theory (Minimax Q-learning) and curriculum learning, provides a dynamic and adaptive mechanism for LLMs to learn sophisticated defense strategies.
• The proposed black-box defense approach is model-agnostic, applicable to various LLMs (e.g., GPT3.5, GPT4) without requiring internal parameter modifications, enhancing its practical utility.

Summary by alphaXiv.