Reproduce “Adapting Large Language Models for Content Moderation: Pitfalls in Data Engineering and Supervised Fine-tuning” on OpenResearch

About this paper

Huan Ma, Changqing Zhang, Huazhu Fu, Peilin Zhao, Bingzhe Wu

Researchers from Tianjin University, IHPC & A*STAR, and Tencent AI Lab adapted Large Language Models (LLMs) for private content moderation using supervised fine-tuning enhanced by Chain of Thought (CoT) reasoning and robust data engineering. The approach improved model robustness and interpretability, enabling fine-tuned Baichuan models to achieve approximately 5% F1 score gains on in-distribution data and competitive zero-shot generalization to novel harmful content compared to commercial LLM APIs like GPT-4.

Key insights

• Incorporating Chain of Thought (CoT) reasoning during supervised fine-tuning is crucial for preventing catastrophic overfitting and enhancing the robustness of LLMs in content moderation.
• Weak supervision combined with data cleaning strategies (e.g., self-recheck and subsequent rejection of inconsistent samples) significantly improves the performance of fine-tuned models.
• The performance benefits from CoT fine-tuning are inherent to the model's capabilities and persist even when explicit reasoning outputs are suppressed during inference, impacting deployment efficiency.

Summary by alphaXiv.