Reproduce “POET: Supporting Prompting Creativity and Personalization with Automated Expansion of Text-to-Image Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Evans Xu Han, Alice Qian Zhang, Haiyi Zhu, Hong Shen, Paul Pu Liang, Jane Hsieh
POET is an interactive text-to-image generation system that automatically identifies and expands homogeneous dimensions in model outputs, then personalizes these expansions using real-time user feedback. This approach enhances creative ideation by producing more diverse image sets, helping users achieve satisfying results in fewer prompting rounds and increasing overall user satisfaction.
Key insights
- "Prompt inversion" effectively identifies implicit homogeneous dimensions in generated images, offering a more scalable and efficient method than direct VLM prompting for multi-image analysis.
- Targeted expansion of these homogeneous dimensions via LLMs significantly diversifies generated outputs while retaining semantic consistency with the user's original prompt.
- Learning from real-time user feedback enables dynamic personalization of expansion strategies, allowing the system to adapt to individual creative preferences and evolving intent.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.