Reproduce “BeautifulPrompt: Towards Automatic Prompt Engineering for Text-to-Image Synthesis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tingfeng Cao, Chengyu Wang, Bingyan Liu, Ziheng Wu, Jinhui Zhu, Jun Huang
BeautifulPrompt, developed by researchers at South China University of Technology and Alibaba Group, automates the generation of high-quality text prompts for diffusion-based text-to-image synthesis models. The system, trained using visual AI feedback, generates prompts that yield images with a PickScore of 20.84 and an Aesthetic Score of 6.52, outperforming existing methods like MagicPrompt and ChatGPT in human preference evaluations.
Key insights
- Visual AI feedback, particularly a combination of human preference (PickScore) and aesthetic quality (Aesthetic Score), can effectively guide language model training for text-to-image prompt optimization.
- Training dedicated reward models that interpret prompt language based on visual outcomes is more stable and effective than directly applying external visual evaluation models for reinforcement learning.
- Automating prompt engineering enhances the usability and accessibility of text-to-image synthesis models, reducing the burden on users.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.