Reproduce “FLUX-Makeup: High-Fidelity, Identity-Consistent, and Robust Makeup Transfer via Diffusion Transformer” on OpenResearch

About this paper

Jian Zhu, Shanyuan Liu, Liuzhuozheng Li, Yue Gong, He Wang, Bo Cheng, Yuhang Ma, Liebucha Wu and 4 more

Researchers at Nanjing University of Science and Technology and 360 AI Research developed FLUX-Makeup, a framework utilizing a Diffusion Transformer to achieve high-fidelity, identity-consistent, and robust makeup transfer. It eliminates the need for auxiliary face-control modules and leverages a meticulously filtered synthetic dataset, outperforming previous methods with higher CLIP-I and SSIM scores while maintaining background consistency.

Key insights

• Advanced Diffusion Transformer architectures can inherently maintain facial identity and background consistency in conditional image generation when properly leveraged.
• A decoupled and targeted feature injection mechanism, like RefLoRAInjector, is essential to transfer makeup styles effectively without compromising source identity or leading to "reference pasting."
• Rigorous multi-stage filtering of synthetically generated training data dramatically improves dataset quality and, consequently, the performance and robustness of makeup transfer models.

Summary by alphaXiv.