Reproduce “Zero-Shot Visual Concept Blending Without Text Guidance” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Hiroya Makino, Takahiro Yamaguchi, Hiroyuki Sakai

Researchers at Toyota Central R&D Labs, Inc. developed a zero-shot Visual Concept Blending method that transfers specific visual attributes from multiple reference images to a source image without requiring text guidance or retraining. This approach operates within a partially disentangled embedding space and demonstrates fine-grained control over blending styles, forms, and abstract concepts, as validated by qualitative results and a user study.

Key insights

  • CLIP embeddings, when linearly projected through IP-Adapter, exhibit partial disentanglement, allowing individual dimensions to capture distinct semantic visual features.
  • Comparing embedding vectors from multiple reference images enables the identification and isolation of both common and unique visual attributes between them.
  • Selective blending of these disentangled features with a source image's embedding facilitates fine-grained and controllable visual concept transfer without additional training or text guidance.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.