Reproduce “FACEMUG: A Multimodal Generative and Fusion Framework for Local Facial Editing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wanglong Lu, Jikai Wang, Xiaogang Jin, Xianta Jiang, Hanli Zhao
FACEMUG presents a multimodal generative and fusion framework that achieves globally-consistent local facial editing by integrating diverse inputs such as sketches, semantic maps, and text. The system effectively preserves unedited regions during modifications and shows improved fidelity and perceptual realism compared to existing methods across various editing tasks.
Key insights
- Integrating diverse modalities into a unified generative latent space provides more precise and flexible control for local facial editing, reducing reliance on extensive paired datasets.
- A self-supervised latent warping mechanism can effectively transfer facial pose in the latent space, preserving identity and other attributes during exemplar-guided transformations.
- An adaptive gated fusion scheme is crucial for blending newly generated features with original facial priors, enabling high-quality, localized modifications that are consistent with unedited image regions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.