Reproduce “Understanding and Mitigating Toxicity in Image-Text Pretraining Datasets: A Case Study on LLaVA” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Karthik Reddy Kanjula, Surya Guthikonda, Nahid Alam, Shayekh Bin Islam
Researchers systematically investigated and quantified toxic content within the LLaVA image-text pretraining dataset, identifying and subsequently removing 7,531 unique toxic image-text pairs. The analysis revealed that sexual content, weapons/substance abuse, and nudity were the most prevalent visual toxic categories in the dataset.
Key insights
- Visual toxicity in the LLaVA dataset was predominantly found in categories such as Sexual Content (33.1%), Weapons/Substance Abuse (22.0%), and Nudity (19.5%).
- Textual toxicity in captions, detected by Toxic-BERT, included general toxicity, obscenity, and insults, with 892 captions flagged with over 80% confidence.
- A systematic, multimodal approach combining specialized visual and textual AI tools can effectively identify and quantify harmful content in large-scale datasets, offering a robust framework for ethical data curation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.