Reproduce “PhotoDoodle: Learning Artistic Image Editing from Few-Shot Pairwise Data” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shijie Huang, Yiren Song, Yuxuan Zhang, Hailong Guo, Xueyin Wang, Mike Zheng Shou, Jiaming Liu
PhotoDoodle presents an instruction-guided framework for artistic image editing that seamlessly integrates decorative elements onto photographs while strictly preserving background content. This method efficiently captures diverse artistic styles from as few as 30-50 examples, outperforming existing state-of-the-art techniques across quantitative metrics and user studies.
Key insights
- Applying identical positional encoding from the source image to both source and noised target latents establishes implicit spatial correspondence, preventing pixel-level misalignment and unwanted background alterations.
- Maintaining a noise-free source image latent representation during iterative denoising effectively preserves high-frequency details and enables targeted content manipulation.
- A two-stage LoRA training approach, with a higher-rank LoRA for general editing and a lower-rank LoRA for few-shot style adaptation, efficiently balances broad capability and specific artistic style capture from limited data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.