Reproduce “Diffusion-based Image Translation using Disentangled Style and Content Representation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Gihyun Kwon, Jong Chul Ye
DiffuseIT is a method for disentangled content and style image translation that guides an unconditional Denoising Diffusion Probabilistic Model (DDPM) with pre-trained Vision Transformer (DINO ViT) features and CLIP. This approach preserves the structural content of a source image while enabling flexible text- or image-guided semantic changes, achieving superior perceptual quality and structural consistency over existing baselines.
Key insights
- Intermediate *keys* from a pre-trained DINO ViT effectively encode structural content, while its `[CLS]` token captures high-level semantic style, enabling robust content-style disentanglement.
- Formulating image translation as an inverse problem with manifold constraints allows for unsupervised and flexible translation using unconditional diffusion models, removing the need for paired training data.
- Actively encouraging semantic change at each diffusion step via a dedicated semantic divergence loss significantly accelerates the translation process without compromising content preservation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.