Reproduce “EditTransfer++: Toward Faithful and Efficient Visual-Prompt-Guided Image Editing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lan Chen, Qi Mao, Yiren Song, Yuchao Gu, Siwei Ma
EditTransfer++ presents a framework to enhance visual-prompt-guided image editing in DiT-based models, improving faithfulness to visual prompts and inference efficiency. It addresses textual dominance and stochastic variability through a progressive training procedure and reduces computational costs with a condition compression and reuse strategy, leading to faster high-resolution generation and better alignment with desired transformations.
Key insights
- Decoupling text conditioning during fine-tuning effectively forces DiT models to learn transformations directly from visual prompts, overcoming the bias of text-to-image pre-training.
- A best-worst contrastive refinement step can actively steer diffusion trajectories, mitigating inherent stochasticity and leading to more consistent and faithful reproduction of visual transformations.
- Reducing token redundancy through condition compression and reusing static conditional features with a causal attention mechanism significantly improves the efficiency of in-context visual learning for high-resolution images without compromising quality.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.