Reproduce “STOP! Benchmarking Large Language Models with Sensitivity Testing on Offensive Progressions” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Robert Morabito, Sangmitra Madhusudan, Tyler McDonald, Ali Emami

Researchers from Brock University introduce the Sensitivity Testing on Offensive Progressions (STOP) dataset, which evaluates Large Language Models' (LLMs) ability to detect bias in progressively offensive scenarios. Fine-tuning models with human judgments from STOP significantly increased LLM engagement and performance on existing bias benchmarks like BBQ, StereoSet, and CrowS-Pairs.

Key insights

  • Large Language Models (LLMs) show significant variability in their ability to detect bias across different severity levels and demographics, with no single model consistently outperforming others.
  • While some LLMs (e.g., Llama 2-70b) show strong 'ideal' sensitivity to bias, Llama 3-70b demonstrated the closest alignment with human judgments, indicating a more 'realistic' sensitivity.
  • Fine-tuning LLMs on human responses from the STOP dataset dramatically improves their engagement and answer rates on established downstream bias evaluation tasks (e.g., 191% increase on StereoSet), suggesting a practical path to more robust and aligned models.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.