Reproduce “Soft-Label Integration for Robust Toxicity Classification” on OpenResearch

About this paper

Zelei Cheng, Xian Wu, Jiahao Yu, Shuo Han, Xin-Qiang Cai, Xinyu Xing

A bi-level optimization framework is introduced to improve robust toxicity classification by integrating crowdsourced annotations via soft-labeling and optimizing weights with Group Distributionally Robust Optimization (GroupDRO). This method consistently outperforms baseline approaches, achieving 78.41% average accuracy and 69.44% worst-group accuracy on a proprietary toxic question dataset, and demonstrates performance comparable to or superior to proprietary LLMs like GPT-4 Turbo for this task.

Key insights

• Learning instance-specific soft-label weights for each annotator effectively captures the diversity and nuances of human judgment regarding toxicity.
• Integrating Group Distributionally Robust Optimization (GroupDRO) with soft-labeling helps mitigate spurious correlations, making the classifier rely on causal features of toxicity.
• The bi-level optimization approach allows for achieving robust performance against out-of-distribution risks by iteratively refining soft labels to improve worst-group accuracy.

Summary by alphaXiv.