Reproduce “Playing the Fool: Jailbreaking LLMs and Multimodal LLMs with Out-of-Distribution Strategy” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Joonhyun Jeong, Seyun Bae, Yeonsung Jung, Jaeryong Hwang, Eunho Yang

Researchers introduced JOOD, a black-box jailbreak framework that bypasses safety alignment in large language models (LLMs) and multimodal LLMs (MLLMs) by transforming harmful inputs into out-of-distribution (OOD) versions. The method achieved up to 74% attack success rate against state-of-the-art models like GPT-4, GPT-4V, GPT-4o, and o1, highlighting a vulnerability in current safety mechanisms to novel input distributions.

Key insights

  • Safety alignment mechanisms in even state-of-the-art LLMs and MLLMs are vulnerable to out-of-distribution inputs, which can be generated through basic data transformations.
  • Introducing semantic ambiguity in text or visual complexity through mixing techniques increases model uncertainty, leading to a higher likelihood of jailbreaking.
  • The degree of semantic dissimilarity between the harmful content and the auxiliary component in mixed inputs is crucial for successfully bypassing refusal mechanisms.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.