Reproduce “ComposeMe: Attribute-Specific Image Prompts for Controllable Human Image Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Guocheng Gordon Qian, Daniil Ostashev, Egor Nemchinov, Avihay Assouline, Sergey Tulyakov, Kuan-Chieh Jackson Wang, Kfir Aberman
A research effort from Snap Inc. introduces ComposeMe, a system for controllable human image generation that allows fine-grained, disentangled manipulation of attributes like face, hair, and clothing from separate reference images. The method achieves high-fidelity outputs and outperforms prior approaches in multi-attribute and multi-identity personalization by learning to naturally compose misaligned inputs.
Key insights
- The 'Multi-Attribute Cross-Reference Training' is crucial for disentangling attributes from confounding factors like pose and expression, effectively eliminating 'copy-paste' artifacts and enabling natural composition from disparate sources.
- Using specialized tokenizers for different attributes (e.g., Omni-ID for face, CLIP-L for hair, SigLIP-L for clothing) significantly enhances the fidelity and consistency of attribute preservation in generated images.
- Attribute-specific image prompting offers a modular paradigm for human image generation, allowing users to flexibly combine and control individual visual traits beyond a single, indivisible identity concept.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.