Reproduce “The effect of fine-tuning on language model toxicity” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Will Hawkins, Brent Mittelstadt, Chris Russell
Research investigates how fine-tuning impacts the toxicity of open large language models, demonstrating that developer-implemented safety mitigations can be inadvertently reversed by subsequent parameter-efficient fine-tuning on benign datasets. The study found that most models exhibited increased toxic outputs, with some categories showing up to a 13.1 percentage point rise in toxicity after such fine-tuning.
Key insights
- Instruction-tuning by model developers effectively reduces initial model toxicity, confirming their stated safety efforts.
- Parameter-efficient fine-tuning (LoRA) on benign datasets can inadvertently increase toxicity and reverse developer-implemented safety mitigations across most open models.
- Toxicity rates in community-tuned models are highly unpredictable, varying significantly based on the base model and fine-tuning intent (e.g., 'uncensoring' vs. multilingual), often lacking transparency in methods.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.