Reproduce “UTDesign: A Unified Framework for Stylized Text Editing and Generation in Graphic Design Images” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yiming Zhao, Yuanpeng Gao, Yuxuan Luo, Jiwei Duan, Shisong Lin, Longfei Xiong, Zhouhui Lian
UTDesign, a collaborative effort by Peking University and Kingsoft Office, introduces a unified framework for high-precision stylized text editing and generation in graphic design images, supporting both English and Chinese scripts with editable RGBA output. The framework significantly outperforms open-source baselines in text accuracy and image quality while approaching proprietary commercial systems.
Key insights
- A progressive training strategy that combines pre-training for editing, feature alignment with an MLLM, and post-training with Direct Preference Optimization (DPO) significantly improves both text accuracy and aesthetic quality in generative design.
- Decomposing text rendering into distinct content and style encoders within a DiT framework, along with a specialized VAE for 4-channel RGBA output, enables precise control and practical editability of stylized text.
- Integrating a capable multi-modal large language model for layout planning, further refined through Group Relative Policy Optimization (GRPO), drastically enhances the coherence and aesthetic placement of generated text in complex graphic designs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.