Reproduce “LLM-based Semantic Augmentation for Harmful Content Detection” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Elyas Meguellati, Assaad Zeghina, Shazia Sadiq, Gianluca Demartini
Researchers from The University of Queensland and Université de Strasbourg developed an LLM-based semantic augmentation framework that cleans noisy text and generates context-rich explanations or explicit triggers for harmful content. This approach led to improved classification performance on tasks such as persuasive meme detection (62.2% H-F1) and toxic comment identification, outperforming zero-shot LLMs and achieving results comparable to human-annotated data at a substantially reduced cost.
Key insights
- Large Language Models (LLMs) are more effective as intelligent data preprocessors and enrichers for complex, high-context classification tasks than as direct zero-shot classifiers.
- Combining LLM-cleaned noisy inputs with LLM-generated semantic explanations or explicit "triggers" significantly enhances the performance of downstream models for harmful content detection.
- A "trigger-based" semantic augmentation strategy can successfully navigate LLM safeguards, allowing critical offensive terms to be preserved and utilized for training models on sensitive content.
- The LLM-based augmentation approach provides performance comparable to human-annotated data at a fraction of the cost, offering a scalable and cost-efficient solution for content moderation datasets.
- Zero-shot LLMs consistently underperform on complex multi-label social media classification tasks compared to fine-tuned or LLM-augmented models, highlighting their limitations in nuanced contexts.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.