Reproduce “DACoN: DINO for Anime Paint Bucket Colorization with Any Number of Reference Images” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kazuma Nagata, Naoshi Kaneko
The DACoN framework leverages frozen DINOv2 features fused with U-Net spatial features to perform anime paint bucket colorization, uniquely supporting any number of reference images. This approach achieves superior segment accuracy, reaching up to 76.66% on keyframe colorization tasks when utilizing multiple references, addressing limitations of prior methods that struggled with compositional variations and restricted reference counts.
Key insights
- Foundation models like DINOv2 can effectively extract robust, part-level semantic features from highly stylized anime line drawings, aiding in complex visual correspondence.
- Removing the constraint on the number of reference images dramatically improves colorization accuracy by providing more comprehensive information for occluded or varied parts.
- A DINO-guided feature consistency loss helps align the model's learned feature similarities with the robust semantic understanding of DINOv2, improving correspondence.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.