Reproduce “AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policies” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yi Zeng, Yu Yang, Andy Zhou, Jeffrey Ziwei Tan, Yuheng Tu, Yifan Mai, Kevin Klyman, Minzhou Pan and 4 more
Researchers developed AIR-BENCH 2024, the first AI safety benchmark directly aligned with 8 government regulations and 16 company policies, featuring 5,694 prompts across 314 risk categories. Evaluations using this benchmark identified considerable safety gaps in leading foundation models, especially regarding "Automated Decision-Making" and "Advice in Heavily Regulated Industries," with top models achieving average refusal rates up to 89%.
Key insights
- No single foundation model consistently adheres to all specified safety guidelines, revealing pervasive and granular safety gaps across the AI landscape, even among top performers.
- Models frequently struggle with specific, policy-critical risk categories such as providing advice in heavily regulated industries and automated decision-making, irrespective of model size or developer.
- The benchmark effectively tracks shifts in model safety performance over time and can detect the impact of developer policy changes, as shown with OpenAI's GPT-3.5 Turbo versions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.