Reproduce “A Theoretical Game of Attacks via Compositional Skills” on OpenResearch

About this paper

Xinbo Wu, Huan Zhang, Abhishek Umrawal, Lav R. Varshney

Researchers introduce a game-theoretic framework to model adversarial prompting against Large Language Models (LLMs) by attackers utilizing compositional skills to hide malicious intent. The framework characterizes inherent attacker advantages due to skill composition scaling and proposes a provably optimal defense strategy that actively misleads attackers, which empirically reduced attack performance by 40.4% to 71.1%.

Key insights

• Equilibrium analysis reveals an inherent attacker advantage: the attacker's gain increases with the size and complexity of the skill composition space, making it difficult for the defender to scale with the rapidly growing number of potential attack vectors.
• The optimal attack strategy involves concentrating probability mass on the specific skill compositions against which the defender is least effective for a given malicious intent.
• A defender can achieve a more advantageous equilibrium by actively misleading the attacker about the actual weakest points, thereby causing the attacker to misallocate its resources and negating the combinatorial advantage of skill composition.

Summary by alphaXiv.