Reproduce “Image Generation from Contextually-Contradictory Prompts” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Saar Huberman, Or Patashnik, Omer Dahary, Ron Mokady, Daniel Cohen-Or
A training-free framework, Stage-Aware Prompting (SAP), improves image generation from text prompts containing contextual contradictions by dynamically adapting prompt conditioning across diffusion denoising stages. Leveraging a Large Language Model for prompt decomposition and staged concept substitutions, the method consistently achieves superior semantic alignment and visual quality for challenging prompts compared to existing baselines.
Key insights
- Diffusion models' coarse-to-fine denoising process allows for introducing semantic information at specific stages: low-frequency structures are set early, while high-frequency details are flexible later.
- Large Language Models possess the world knowledge and reasoning capabilities necessary to identify contextual contradictions and strategically decompose prompts into contradiction-free, stage-appropriate proxies.
- Temporarily substituting problematic concepts with placeholder alternatives enables the model to establish a coherent overall scene structure and object placement before introducing potentially conflicting final identities.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.