Reproduce “TextSculptor: Training and Benchmarking Scene Text Editing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yiheng Lin, Siyu Jiao, Xiaohan Lan, Wei Zhou, Qi She, Fei Yu, Heyun Chen, Zhengwei Wang and 7 more
TextSculptor addresses limitations in scene text editing by introducing TextSculpt-Data, a 3.2 million sample high-fidelity dataset, and TextSculpt-Bench, a standardized benchmark for evaluation. The framework enables training a model that achieves an overall average score of 0.69 on the benchmark, demonstrating balanced performance across text accuracy, visual quality, and background preservation.
Key insights
- Combining VLM-based caption rewriting, high-quality image synthesis, and programmatic text compositing can generate extensive, high-fidelity data crucial for robust scene text editing, ensuring both textual accuracy and pixel-level background consistency.
- Explicit supervision through the inclusion of 'distraction' text (multiple text instances) in training samples significantly enhances a model's ability to handle cluttered text layouts and improves visual quality.
- A multi-dimensional benchmark evaluating text correctness, visual plausibility, and background preservation simultaneously is essential to accurately diagnose specific limitations and guide improvements in scene text editing models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.