Reproduce “HairCLIP: Design Your Hair by Text and Reference Image” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Tianyi Wei, Dongdong Chen, Wenbo Zhou, Jing Liao, Zhentao Tan, Lu Yuan, Weiming Zhang, Nenghai Yu

HairCLIP presents a unified framework for realistic hair manipulation that accepts both text descriptions and reference images as input. It effectively disentangles hair attributes and preserves identity, producing high-fidelity edits while outperforming previous text-driven and image-based hair transfer methods in visual quality and user preference.

Key insights

  • StyleGAN's latent space layers exhibit a semantic hierarchy, where coarse layers control high-level attributes (e.g., hairstyle) and fine layers control low-level details (e.g., hair color).
  • CLIP's shared image-text embedding space allows for a unified representation of diverse multi-modal inputs (text and reference images) for guiding image generation.
  • Measuring semantic similarity using CLIP's image embeddings is robust for hair transfer, enabling manipulation even with significant pose or structural misalignment between input and reference images.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.