Reproduce “A Systematic Review of Open Datasets Used in Text-to-Image (T2I) Gen AI Model Safety” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rakeen Rouf, Trupti Bavalatti, Osama Ahmed, Dhaval Potdar, Faraz Jawed
A systematic review examined eight open-source datasets crucial for Text-to-Image (T2I) generative AI model safety, revealing a severe overrepresentation of sexual content and critical deficiencies in harm coverage, diversity, and label quality. The study provides a comprehensive analysis of dataset limitations and outlines directions for developing more robust safety data.
Key insights
- Current T2I safety datasets exhibit a severe imbalance in harm coverage, with an overemphasis on 'Sexual Content' and underrepresentation of other critical harms.
- There are significant deficiencies in prompt diversity, including an overwhelming dominance of English and limited semantic/syntactic variability.
- Label quality is inconsistent across datasets and harm types, and synthetic data artifacts (e.g., 'gibberish' substrings) indicative of model collapse were observed.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.