Reproduce “DiffusionAgent: Navigating Expert Models for Agentic Image Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jie Qin, Jie Wu, Weifeng Chen, Yueming Lyu
DiffusionAgent routes natural-language image requests to existing specialist models. A language model parses each request, searches a subject-and-style model tree, uses an offline database of reward-scored generations to guide selection, and extends the prompt before generation. In the reported SD1.5 comparison, it scores 0.63 on ImageReward and 5.70 on aesthetic score, versus 0.28 and 5.26 for SD1.5; evaluation details remain limited.
Key insights
- Large Language Models can effectively serve as cognitive engines to orchestrate and manage a wide range of specialized generative models, enhancing prompt comprehension and expert model utilization.
- A hierarchical organization (Tree-of-Thought) combined with an offline human-feedback-aligned database is effective for efficiently routing diverse prompts to the most appropriate expert models.
- LLM-powered prompt extension significantly improves the aesthetic quality and detail of generated images by providing more specific and contextually rich input to the selected expert diffusion models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.