Reproduce “PartEdit: Fine-Grained Image Editing using Pre-Trained Diffusion Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Aleksandar Cvejic, Abdelrahman Eldesokey, Peter Wonka

King Abdullah University of Science and Technology researchers developed PartEdit, a method for fine-grained, text-based image editing that modifies specific object parts using pre-trained diffusion models. It learns specialized textual tokens for parts and employs a three-path denoising process with adaptive blending, achieving precise and seamless edits that were preferred by users over existing techniques.

Key insights

  • Efficiently expanding the semantic understanding of pre-trained diffusion models for specific object parts is achievable by optimizing a small number of textual tokens.
  • Cross-attention maps from intermediate UNet layers provide rich, localizable semantic information that can be effectively guided by part tokens for precise editing regions.
  • Combining adaptive thresholding of attention maps with feature blending during inference is crucial for producing high-quality, seamless, and localized fine-grained edits.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.