Reproduce “Adversarial Attack on Black-Box Multi-Agent by Adaptive Perturbation” on OpenResearch

About this paper

Jianming Chen, Yawen Wang, Junjie Wang, Xiaofei Xie, Yuanzhe Hu, Qing Wang, Fanjiang Xu

AdapAM, an adaptive adversarial attack framework, enables black-box multi-agent systems (MAS) to be compromised by strategically perturbing observations of a single chosen agent. It consistently reduced MAS win rates, often to 0%, while maintaining superior stealthiness with minimal perturbation magnitudes and low detectability across diverse multi-agent benchmarks.

Key insights

• Effective black-box adversarial attacks on MAS can be achieved by adaptively selecting a single critical agent and inducing specific malicious actions through targeted observation perturbations.
• Proxy models, trained via imitation learning, can successfully bridge the gap between the need for white-box information to generate effective perturbations and the constraints of black-box multi-agent environments.
• A hierarchical design for adaptive selection, combined with a proxy-based perturbation generation, allows for high attack effectiveness while maintaining stealthiness by minimizing perturbation magnitude and targeting only one agent.

Summary by alphaXiv.