Reproduce “When Safety Becomes An Outlier: Understanding the Retention of LLM Safety Behaviors” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Binchi Zhang, Hadi Abdullah, Yiwei Cai, Jundong Li
NEST uses prompt-grounded safety responses as fine-tuning targets to preserve refusals during benign adaptation. For supervised fine-tuning, it generates contextual answers that address harmful intent and redirect. On Llama-3-8B, after 10,000 benign examples, NEST’s harmful rate is 60.61%, versus 79.83% for fixed refusals. This improves retention, not immunity, in tested open 7B/8B models and setups.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.