Reproduce “Character-Aware Models Improve Visual Text Rendering” on OpenResearch

About this paper

Rosanne Liu, Dan Garrette, Chitwan Saharia, William Chan, Adam Roberts, Sharan Narang, Irina Blok, RJ Mical and 2 more

Research from Google demonstrates that incorporating character-aware text encoders substantially improves the ability of text-to-image models to render legible visual text. This approach yields gains of 15-30+ percentage points in spelling accuracy on a new benchmark and improved human preference for creative text rendering compared to models using character-blind encoders.

Key insights

• Character-blind language models only acquire strong spelling abilities at exceptionally large scales and primarily for English, while character-aware models excel at much smaller scales and across languages.
• The core limitation in visual text rendering stems from the text encoder's lack of character-level awareness, leading to distinct error patterns such as semantic errors from character-blind models versus layout issues from character-aware models.
• A hybrid approach, combining both token-level (character-blind) and character-level text embeddings, offers the most effective solution for robust text rendering while maintaining general image quality.

Summary by alphaXiv.