Reproduce “DoveNet: Deep Image Harmonization via Domain Verification” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Wenyan Cong, Jianfu Zhang, Li Niu, Liu Liu, Zhixin Ling, Weiyuan Li, Liqing Zhang

Researchers from Shanghai Jiao Tong University introduced DoveNet, a deep image harmonization method leveraging a novel domain verification discriminator, and released iHarmony4, the first large-scale public dataset for this task. DoveNet consistently achieved state-of-the-art performance in both objective metrics and subjective human evaluations.

Key insights

  • Image harmonization can be effectively re-conceptualized as a domain translation problem, where the goal is to align the foreground's visual domain with the background's.
  • Explicitly training a discriminator to verify domain consistency between foreground and background regions (via partial convolutions for feature extraction) provides a powerful signal for improving harmonization.
  • A carefully constructed, diverse, and large-scale dataset is crucial for advancing deep learning in image harmonization, enabling models to learn complex relationships.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.