Reproduce “Adversarial Poetry as a Universal Single-Turn Jailbreak Mechanism in Large Language Models” on OpenResearch

About this paper

Piercosma Bisconti, Matteo Prandi, Federico Pierucci, Francesco Giarrusso, Marcantonio Bracale Syrnikov, Marcello Galisai, Vincenzo Suriani, Olga Sorokoletova and 2 more

Researchers at DEXAI – Icaro Lab and Sapienza University of Rome introduce "Adversarial Poetry" as a single-turn jailbreak mechanism for Large Language Models. This method, involving poetic reformulation of harmful prompts, consistently bypasses safety mechanisms across 25 frontier LLMs, increasing Attack Success Rates by up to 18 times compared to original prose prompts, and reveals a "scale paradox" where smaller models sometimes exhibit greater resilience.

Key insights

• Poetic form alone can systematically degrade LLM refusal behavior, indicating a vulnerability in how safety mechanisms process figurative or stylized language.
• The observed vulnerability generalizes across diverse model families and heterogeneous risk domains, suggesting a systemic limitation in existing alignment pipelines.
• A "scale paradox" was identified, where smaller models often exhibited higher refusal rates than their larger, more capable counterparts within the same provider families.

Summary by alphaXiv.