Reproduce “ConceptBed: Evaluating Concept Learning Abilities of Text-to-Image Diffusion Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Maitreya Patel, Tejas Gokhale, Chitta Baral, Yezhou Yang

ConceptBed evaluates personalized text-to-image methods by testing whether images preserve concepts from reference images while following compositions. It contains 284 concepts and about 33,000 composite prompts, and introduces CCD, which compares oracle confidence on generated and real images. Across methods, Textual Inversion and Custom Diffusion trade off strengths; CCD correlates with ratings (−0.9892 and −0.7342), but only within these tasks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.