Reproduce “Add-it: Training-Free Object Insertion in Images With Pretrained Diffusion Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yoad Tewel, Rinon Gal, Dvir Samuel, Yuval Atzmon, Lior Wolf, Gal Chechik
A new training-free method named ADD-IT enables seamless and contextually plausible object insertion into images based on textual instructions. It achieves state-of-the-art performance, significantly improving object placement plausibility on a new benchmark by raising the affordance score from 47.4% to 82.8%.
Key insights
- Precise, training-free object insertion is achievable by intelligently controlling and extending the attention mechanisms of pre-trained text-to-image diffusion models.
- Dynamically balancing the influence of the source image and the target object prompt is critical for natural object placement and scene preservation.
- "Affordance" (plausible object placement) is a key bottleneck in object insertion, and its quantitative evaluation is crucial for advancing the field.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.