Reproduce “ColorConceptBench: A Benchmark for Probabilistic Color-Concept Understanding in Text-to-Image Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chenxi Ruan, Yihan Hou, Yu Xiao, Guosheng Hu, Wei Zeng
ColorConceptBench tests whether text-to-image models adapt color distributions to visual-state and emotional modifiers, using designers’ colorings of sketches as references. It compares base and modified concepts with Earth Mover’s Distance across nine models. Mean shifts reach 0.769 of the human shift for visual states and 0.716 for emotions; these ratios measure magnitude, not directional accuracy, on tangible concepts with region-limited annotations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.