Reproduce “ConceptLab: Creative Concept Generation using VLM-Guided Diffusion Prior Constraints” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Elad Richardson, Kfir Goldberg, Yuval Alaluf, Daniel Cohen-Or
ConceptLab, developed at Tel Aviv University, introduces a method for generating unique, never-before-seen concepts within broad categories using VLM-guided diffusion prior constraints. The approach enables the creation of diverse novel objects and styles, outperforming baselines in distinctness from known examples and facilitating the mixing of concepts.
Key insights
- Applying constraints in the Diffusion Prior's output space is more effective for guiding novel concept generation than direct CLIP or image space, ensuring consistent properties.
- Dynamically expanding negative constraints using a Vision-Language Model (BLIP-2) is crucial for ensuring generated concepts diverge from known instances and promoting diversity.
- The framework effectively blends multiple concepts to create plausible hybrids, enabling an "evolutionary generation" of novel entities.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.