Reproduce “Red Teaming Language Models with Language Models” on OpenResearch

About this paper

Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese and 1 more

The authors present language-model red teaming, in which a red language model generates test prompts, a target chatbot answers, and a classifier flags candidate failures. Across offensive-language, leakage, bias, and dialogue tests, the method scales beyond manual prompts. In 500,000 zero-shot questions, 18,444 replies were flagged offensive, but results depend on classifier judgments and the 280B-parameter DPG target.

Key insights

• Language models can effectively red team other language models, offering a scalable, automated, and versatile approach to discovering diverse harmful behaviors.
• Different test case generation strategies, from zero-shot prompting to reinforcement learning, offer distinct trade-offs between diversity and difficulty in eliciting harmful responses.
• The method is versatile and capable of uncovering various types of harms beyond offensive content, including data leakage, generation of private contact information, and subtle distributional biases.

Summary by alphaXiv.