Reproduce “Advancing Content Moderation: Evaluating Large Language Models for Detecting Sensitive Content Across Text, Images, and Videos” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Nouar AlDahoul, Myles Joshua Toledo Tan, Harishwar Reddy Kasireddy, Yasir Zaki
This research rigorously evaluates large language models (LLMs) and multimodal LLMs for sensitive content detection, demonstrating that general-purpose models like GPT-4o and Gemini 1.5 often outperform traditional methods and dedicated moderation tools across text, images, and videos. The findings show improved accuracy and better balancing of false positive and false negative rates, with Gemini 1.5 Pro achieving 95.5% accuracy for video violence detection.
Key insights
- General-purpose multimodal LLMs consistently outperformed traditional techniques and often surpassed dedicated LLM-based moderation solutions, especially in nuanced visual abuse categories and in balancing false positive and false negative rates.
- Dedicated moderation models exhibited limitations in detecting specific categories of abuse (e.g., alcohol, drug, child abuse) or certain graphic violence types not explicitly covered by their predefined taxonomies.
- LLMs demonstrated inconsistency in prediction accuracy across different types of inappropriate content (e.g., violence vs. nudity) and between models, highlighting the need for careful model selection based on specific moderation requirements.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.