Reproduce “HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Wei Wei, Tingbo Hou, Yael Pritch, Neal Wadhwa, Michael Rubinstein and 1 more
HyperDreamBooth enables rapid, high-fidelity personalization of text-to-image diffusion models, creating subject-specific models in approximately 20 seconds while reducing model size by 10,000 times compared to prior methods. It achieves superior subject identity preservation and style diversity compared to existing fast personalization techniques.
Key insights
- Decomposing LoRA residuals into a product of small, learnable matrices and fixed, random orthogonal bases dramatically reduces the number of trainable parameters for extreme parameter efficiency.
- A HyperNetwork with a transformer decoder can effectively predict diffusion model weights, modeling inter-layer dependencies and enabling rapid personalization without subject-specific fine-tuning.
- Combining a predictive HyperNetwork for quick initialization with a brief, rank-relaxed fine-tuning step offers a powerful hybrid approach to balance speed, efficiency, and high-fidelity detail capture.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.