Reproduce “TypeScore: A Text Fidelity Metric for Text-to-Image Generative Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Georgia Gabriela Sampaio, Ruixiang Zhang, Shuangfei Zhai, Jiatao Gu, Josh Susskind, Navdeep Jaitly, Yizhe Zhang
Researchers from Apple introduce TYPESCORE, a novel metric specifically designed to assess the fidelity of embedded text within images generated by text-to-image models. TYPESCORE consistently shows higher alignment with human judgments on text and overall image quality compared to existing metrics like CLIPScore, and reveals that Ideogram achieves the highest text fidelity among tested state-of-the-art models.
Key insights
- The ability to accurately render embedded text serves as a strong proxy for a generative model's broader fine-grained control and instruction-following capabilities.
- Richer, recaptioned prompts containing detailed stylistic and contextual information can lead to improved text fidelity in generated images.
- The relative ranking of text-to-image models by TYPESCORE remains robust even when using different, less accurate text extraction models like LLaVA-NeXT, as long as error levels are consistent across evaluated images.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.