Reproduce “On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hao Sun, Guangxuan Xu, Jiawen Deng, Jiale Cheng, Chujie Zheng, Hao Zhou, Nanyun Peng, Xiaoyan Zhu and 1 more
Researchers from Tsinghua University, UCLA, and Tencent introduced a comprehensive taxonomy for context-sensitive dialogue unsafety, developed the DIASAFETY dataset, and established a benchmark for evaluating conversational models. Their RoBERTa-based classifier achieved an 84.0% F1-score in detecting nuanced unsafe dialogue, revealing that existing state-of-the-art models struggle significantly with context-dependent safety issues, often regardless of their scale.
Key insights
- Context is critical for detecting nuanced unsafe behaviors in dialogue, with a substantial 13.4% F1-score drop observed when context was removed from the detection task.
- Existing utterance-level safety tools (e.g., Perspective API, Detoxify) and general context-aware models perform poorly on context-sensitive unsafety, demonstrating their limitations.
- Larger conversational models do not consistently correlate with safer responses, challenging the assumption that increased scale automatically leads to more responsible behavior.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.