Reproduce “CoCoEdit: Content-Consistent Image Editing via Region Regularized Reinforcement Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuhui Wu, Chenxi Xie, Ruibin Li, Liyi Chen, Qiaosi Yi, Lei Zhang
Researchers at The Hong Kong Polytechnic University and OPPO Research Institute developed CoCoEdit, a post-training framework that employs region-regularized reinforcement learning for content-consistent image editing. It integrates MLLM feedback with pixel-level similarity rewards and region-based regularizers to enhance desired edits while preserving non-edited areas, leading to significant improvements in content consistency metrics like PSNR (+2.8dB for Qwen-Image-Edit on GEdit-Bench) without compromising editing quality.
Key insights
- Combining high-level semantic feedback from MLLMs with fine-grained pixel-level similarity rewards provides a more comprehensive and balanced signal for training content-consistent image editing models.
- Spatially aware regularizers that enforce distinct constraints on edited versus non-edited regions are crucial for overcoming the limitations of global, scalar rewards and effectively improving content preservation.
- A high-quality, mask-annotated dataset specifically tailored for local editing tasks is essential for enabling reinforcement learning frameworks to learn and maintain regional consistency during image manipulation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.