Reproduce “UniCon: Unidirectional Information Flow for Effective Control of Large-Scale Diffusion Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Fanghua Yu, Jinjin Gu, Jinfan Hu, Zheyuan Li, Chao Dong
Developed by researchers at Shenzhen Institutes of Advanced Technology, UniCon proposes a unidirectional information flow design for training control adapters for large-scale diffusion models, reducing training VRAM by one-third and accelerating training by 2.3 times while improving generative quality and control precision across various tasks.
Key insights
- Decoupling gradient computation from the large pre-trained diffusion model through a unidirectional information flow substantially boosts training efficiency for control adapters.
- Empowering the adapter to directly generate the final output, rather than merely modifying intermediate features, allows for enhanced expressive power, leading to superior generative quality and finer control.
- The design of the adapter, specifically mirroring the base model's architecture and employing a zero-initialized feature transform (ZeroFT) connector, is crucial for stable and effective training across various diffusion model types.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.