Reproduce “VLM-Guided Adaptive Negative Prompting for Creative Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Shelly Golan, Yotam Nitzan, Zongze Wu, Or Patashnik

Researchers from Technion, Adobe Research, and Tel Aviv University introduce a VLM-Guided Adaptive Negative-Prompting method that dynamically steers text-to-image models to generate truly novel visual concepts while maintaining semantic validity. This training-free approach consistently produces creative outputs, validated by user studies and quantitative metrics showing a superior balance of novelty and category coherence compared to existing methods.

Key insights

  • Adaptive, real-time VLM feedback is crucial for steering generative models towards exploratory creativity, outperforming static or non-adaptive negative prompting strategies.
  • The 'Momentum Effect' in diffusion allows VLM guidance to be limited to the first 10-15 denoising steps, effectively reducing computational overhead without significantly compromising creative quality.
  • The method successfully balances the generation of novel concepts with maintaining semantic validity by dynamically identifying and pushing away from conventional interpretations using negative prompting.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.