Reproduce “Mixture of Global and Local Experts with Diffusion Transformer for Controllable Face Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xuechao Zou, Shun Zhang, Xing Fu, Yue Li, Kai Li, Yushe Cao, Congyan Lang, Pin Tao and 1 more

Face-MoGLE is a Diffusion Transformer-based framework that enables high-quality and controllable face generation from text and semantic masks. It integrates a Mixture of Global and Local Experts architecture with a dynamic, diffusion-aware gating network to decouple semantic controls, leading to improved photorealism, precise attribute manipulation, and strong zero-shot generalization while outperforming prior state-of-the-art methods across various metrics.

Key insights

  • Decoupling semantic controls from the generation process via a dedicated mask encoder and dynamic gating allows for more precise attribute manipulation without compromising photorealism.
  • A hierarchical expert fusion approach, combining global structural priors with local, region-specific semantics, significantly enhances both overall image quality and semantic alignment.
  • The use of a Diffusion Transformer backbone, combined with a dynamic, spatially-aware gating mechanism, improves long-range dependency modeling and adaptability to complex multimodal conditional inputs.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.