Reproduce “Prompt Expansion for Adaptive Text-to-Image Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Siddhartha Datta, Alexander Ku, Deepak Ramachandran, Peter Anderson
A new Prompt Expansion framework by Google Research enhances text-to-image generation by automatically transforming simple user queries into multiple detailed prompts. This approach results in images that are more diverse and aesthetically preferred by human raters, while significantly reducing the need for manual prompt engineering.
Key insights
- Transforming simple user queries into multiple detailed prompts with a text-to-text model significantly enhances the creative output of text-to-image models.
- Re-fine-tuning the prompt expansion model using feedback from the downstream text-to-image model (Imagen) is crucial for generating prompts that lead to high aesthetic quality.
- Introducing diversity in the text space via prompt expansion offers a more interpretable and controllable method for varied image generation compared to adjusting sampling parameters in image models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.