Reproduce “Enhancing Jailbreak Attacks on LLMs via Persona Prompts” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zheng Zhang, Peilin Zhao, Deheng Ye, Hao Wang
Researchers from The Hong Kong University of Science and Technology (Guangzhou) and Tencent demonstrated that persona prompts can weaken large language model safety defenses, developing a genetic algorithm to automatically generate such prompts that reduce refusal rates by 50-70% and enhance existing jailbreak attack success by 10-30% across various LLMs. The study provides insights into how these prompts shift LLM attention, making models more susceptible to generating harmful content.
Key insights
- Persona prompts alone can significantly lower LLM refusal rates, creating a vulnerable context that is not necessarily immediately exploited for harmful content.
- A strong synergistic effect exists where persona prompts, when combined with other jailbreak methods, substantially increase the overall attack success rate by weakening refusal mechanisms.
- Effective persona prompts shift LLM attention from sensitive terms to stylistic instructions, enabling compliance, and are most effective when placed as system prompts.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.