Reproduce “GRP-Obliteration: Unaligning LLMs With a Single Unlabeled Prompt” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Mark Russinovich, Yanan Cai, Keegan Hines, Giorgio Severi, Blake Bullwinkel, Ahmed Salem

GRP-Obliteration, developed by Microsoft, presents a method for reliably removing safety alignment from LLMs and diffusion models using as little as a single unlabeled prompt, while largely preserving the model's original utility. This approach consistently outperformed existing un-alignment techniques across diverse models and benchmarks, demonstrating the fragility of current AI safety mechanisms.

Key insights

  • Current safety alignment mechanisms exhibit extreme fragility, as robust un-alignment can be achieved with an unprecedentedly small amount of data, sometimes a single unlabeled prompt, without significantly compromising model utility.
  • Un-alignment achieved by GRP-Oblit generalizes broadly across diverse harm categories in LLMs, suggesting it targets underlying safety behaviors rather than narrow refusal patterns, and also alters the model's internal perception of harmfulness.
  • The vulnerability to parameter-level un-alignment is not unique to LLMs but extends to other foundation models like diffusion-based text-to-image systems, indicating a broader property of modern AI pipelines.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.