Reproduce “ReEdit: Multimodal Exemplar-Based Image Editing with Diffusion Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ashutosh Srivastava, Tarun Ram Menta, Abhinav Java, Avadhoot Jadhav, Silky Singh, Surgan Jandial, Balaji Krishnamurthy
ReEdit enables image transformations by learning a desired edit from an exemplar before-and-after image pair, outperforming text-based and other exemplar-based methods in applying the demonstrated style, object, or background changes to new target images while preserving content.
Key insights
- Capturing edit intent is most effective when combining image embedding differences for visual transformations with natural language understanding (via LLaVA) for complex object changes.
- DDIM inversion, feature injection, and self-attention injection are crucial for maintaining the non-edited regions and overall structure of the target image during transformation.
- The modular design of the framework, integrating CLIP embeddings, LLaVA, and diffusion model injection techniques, allows for a robust and balanced approach to diverse image editing scenarios.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.