Reproduce “GenPilot: A Multi-Agent System for Test-Time Prompt Optimization in Image Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Wen Ye, Zhaocheng Liu, Yuwei Gui, Tingyu Yuan, Yunyue Su, Bowen Fang, Chaoyang Zhao, Qiang Liu and 1 more

GenPilot, a multi-agent system leveraging multimodal large language models, introduces a test-time prompt optimization strategy to enhance text-to-image consistency for complex prompts. It achieves consistent performance gains across diverse text-to-image models, improving image quality and semantic alignment without requiring model-specific training.

Key insights

  • Multimodal Large Language Models can effectively act as intelligent agents for systematic error analysis, fine-grained verification, and dynamic prompt refinement in text-to-image synthesis.
  • An integrated strategy combining VQA and caption-based error detection provides more comprehensive and accurate identification of inconsistencies between images and complex prompts.
  • Iterative optimization in the input text space, guided by a test-time MLLM scorer, clustering, and memory, enables efficient exploration and convergence towards better prompts.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.