Reproduce “VRSA: Jailbreaking Multimodal Large Language Models through Visual Reasoning Sequential Attack” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Shiji Zhao, Shukun Xiong, Yao Huang, Yan Jin, Zhenyu Wu, Jiyang Guan, Ranjie Duan, Jialing Tao and 2 more

A novel jailbreaking method, VRSA, exploits multimodal large language models' visual reasoning capabilities by presenting harmful intent as optimized, semantically coherent image sequences. This approach achieved average attack success rates up to 76.57% on open-source models and 62.57% on commercial closed-source MLLMs like GPT-4o, significantly outperforming existing baselines.

Key insights

  • MLLMs' sophisticated visual reasoning, often facilitated by Chain-of-Thought mechanisms, can be exploited by carefully structured visual sequences that gradually reveal harmful intent.
  • Optimizing the scene's rationality and ensuring semantic continuity across a sequence of images are critical for effectively bypassing MLLM safety alignments.
  • The synergistic combination of rich pictorial content from generated images and explicit textual typography within those images significantly enhances the effectiveness of multimodal jailbreak attacks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.