Reproduce “RePainter: Empowering E-commerce Object Removal via Spatial-matting Reinforcement Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zipeng Guo, Lichen Ma, Xiaolong Fu, Gaojing Zhou, Lan Yang, Yuchen Zhou, Linkai Liu, Yu He and 8 more

RePainter introduces a reinforcement learning framework for e-commerce object removal, leveraging spatial-matting trajectory refinement and a local-global composite reward mechanism. The method generates visually seamless and semantically coherent images, outperforming existing state-of-the-art inpainting techniques across multiple quantitative and qualitative evaluations.

Key insights

  • Modulating attention with spatial-matting during the image generation process enhances exploration efficiency and reduces unwanted object hallucination in reinforcement learning.
  • A comprehensive, composite reward function integrating global structure, local reconstruction, and semantic validity is crucial for guiding generative models to achieve holistic image quality and prevent suboptimal outcomes.
  • Group Relative Policy Optimization (GRPO) can be effectively applied to flow-based generative models for fine-tuning complex visual content removal tasks, especially when combined with improved sampling strategies and robust reward design.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.