Reproduce “Region-Wise Correspondence Prediction between Manga Line Art Images” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yingxuan Li, Jiafeng Mao, Qianru Qiu, Yusuke Matsui

This research introduces a Transformer-based framework to predict region-wise correspondences between raw, unannotated manga line art images, addressing the challenge of sparse visual cues. The method includes a novel automatic data annotation pipeline and achieves robust patch-level similarity learning and coherent region matching, improving upon existing baselines.

Key insights

  • A Transformer architecture can effectively learn structural relationships and cross-image correspondences in highly sparse and abstract visual data like manga line art, even without traditional cues like color or texture.
  • Combining patch-level similarity learning with a post-processing step that incorporates image gradient information enables the generation of semantically coherent regions aligned with line art strokes.
  • Large-scale, automatically generated pseudo-labeled datasets are crucial for training deep learning models on specialized visual domains lacking extensive manual annotations, improving model performance and generalization.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.