Reproduce “Exploring Multimodal Challenges in Toxic Chinese Detection: Taxonomy, Benchmark, and Findings” on OpenResearch

About this paper

Shujian Yang, Shiyao Cui, Chuanrui Hu, Haicheng Wang, Tianwei Zhang, Minlie Huang, Jialiang Lu, Han Qiu

A study from Shanghai Jiao Tong University, Tsinghua University, and others presents a comprehensive taxonomy of multimodal Chinese toxicity perturbations and curates CNTP, a benchmark dataset. Their evaluation demonstrates that state-of-the-art large language models are highly susceptible to these subtle manipulations, and common enhancement techniques can cause an overcorrection, misclassifying benign content.

Key insights

• Perturbations leveraging homophones and Pinyin initials are particularly effective at confusing state-of-the-art LLMs, significantly reducing detection rates.
• All benchmarked LLMs show a substantial decrease in detection accuracy on perturbed Chinese content, demonstrating a general vulnerability to these subtle manipulations.
• Cost-effective enhancement strategies like ICL and SFT can improve detection but cause a drastic increase in misclassifying benign content as toxic, revealing an "overcorrection" phenomenon rather than genuine semantic understanding.

Summary by alphaXiv.