Reproduce “MUSE: Multi-Subject Unified Synthesis via Explicit Layout Semantic Expansion” on OpenResearch

About this paper

Fei Peng, Junqiang Wu, Yan Li, Tingting Gao, Di Zhang, Huiyuan Fu

MUSE, developed by researchers at Beijing University of Posts and Telecommunications and Kuaishou Technology, presents a framework for generating multi-subject images with precise layout control and identity preservation using text-to-image diffusion models. The approach improves layout success rates by 10.3% on the MIG Bench and maintains subject fidelity in complex scenarios, achieving 3.9 to 6.5 times faster inference than prior methods.

Key insights

• Integrating explicit layout conditions by concatenating them with text semantics in cross-attention effectively resolves control collision and improves spatial adherence in generated images.
• Decoupling the learning objectives for layout control and subject identity preservation into a progressive two-stage training process mitigates optimization conflicts and yields more stable, high-quality results.
• Using multiple image feature tokens and a consistent additive bounding box integration method for subject grounding improves the fidelity and internal consistency of generated subjects.

Summary by alphaXiv.