Reproduce “Scalable Question-Centric Text-to-Image Evaluation: Reliable Ranking, Fine-Grained Diagnosis, and Cost-Aware Routing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shaoan Zhao, Fang Zhao, Xueqiang Guo, Xinpei Su, Huanlin Gao, Qiang Hui, Ting Lu, Fuyuan Shi and 4 more
QC-T2I-Bench evaluates text-to-image generators by checking prompt requirements as atomic questions, recording prerequisite links, and reusing question-level evidence for diagnosis and routing. Across evaluated models, exact completion averages 80.7% for components with two capability tags and 37.2% for those with seven or more. This measures accumulating joint difficulty, not proof of interaction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.