Reproduce “EditMGT: Unleashing Potentials of Masked Generative Transformers in Image Editing” on OpenResearch

About this paper

Wei Chow, Linfeng Li, Lingdong Kong, Zefeng Li, Qi Xu, Hang Song, Tian Ye, Xian Wang and 8 more

EditMGT introduces the first framework leveraging Masked Generative Transformers (MGTs) for general image editing, explicitly addressing the "editing leakage" problem. This is achieved through adaptive localization via multi-layer attention consolidation and explicit region preservation using region-hold sampling, resulting in state-of-the-art image similarity and 6x faster editing speeds for 1024x1024 images with a compact 960M parameter model.

Key insights

• The localized decoding paradigm and token-flipping nature of MGTs inherently provide a robust mechanism for explicit preservation of non-target regions, offering a fundamental solution to editing leakage.
• Cross-attention maps within MGTs can be effectively consolidated and refined to provide accurate, adaptive localization signals for editing, reducing reliance on external segmentation tools.
• A pre-trained text-to-image MGT can be efficiently adapted for diverse image editing tasks through attention injection and targeted inference strategies, maintaining high quality without substantial additional parameters.

Summary by alphaXiv.