Reproduce “JPRO: Automated Multimodal Jailbreaking via Multi-Agent Collaboration Framework” on OpenResearch

About this paper

Yuxuan Zhou, Yang Bai, Kuofeng Gao, Tao Dai, Shu-Tao Xia

Researchers at Tsinghua University and ByteDance introduced JPRO, an automated multi-agent collaborative framework for black-box VLM jailbreaking. The system successfully bypasses safety mechanisms in proprietary models like GPT-4o and open-source VLMs, achieving attack success rates up to 75.50% on Qwen2.5-VL and demonstrating improved attack diversity and transferability over existing methods.

Key insights

• A multi-agent collaborative framework (comprising Planner, Attacker, Modifier, and Verifier agents) can effectively automate black-box VLM jailbreaking, surpassing the limitations of manual or white-box methods.
• Maintaining malicious intent in multi-turn multimodal attacks necessitates explicit strategy guidance and iterative verification to counteract the degradation of harmfulness.
• Addressing the "Cross-Modal Expression Gap" through real-time semantic rectification of visual inputs is crucial for achieving successful and precisely targeted multimodal jailbreaks.

Summary by alphaXiv.