Reproduce “T2I-RiskyPrompt: A Benchmark for Safety Evaluation, Attack, and Defense on Text-to-Image Model” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Chenyu Zhang, Tairen Zhang, Lanjun Wang, Ruidong Chen, Wenhui Li, Anan Liu

T2I-RiskyPrompt is introduced as a comprehensive benchmark for evaluating text-to-image (T2I) model safety, featuring a hierarchical risk taxonomy, over 6,000 effective risky prompts, and a reason-driven detection method. The benchmark reveals that advanced T2I models exhibit higher risks and that current defense mechanisms struggle against diverse content and sophisticated attacks, highlighting persistent safety vulnerabilities.

Key insights

  • T2I models with stronger generative capabilities tend to exhibit greater safety risks across categories, implying that improved instruction-following also enables more effective generation of harmful content.
  • Defending against risk content with diverse visual manifestations, such as copyright infringement, presents a significant challenge for all tested defense methods due to the inherent semantic unrelatedness of concepts.
  • There is a persistent trade-off between defense strength and generation quality, where stronger protective measures often correlate with more substantial degradation in the overall quality of generated images.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.