Reproduce “Language-Driven Image Style Transfer” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tsu-Jui Fu, Xin Eric Wang, William Yang Wang
A framework named Contrastive Language Visual Artist (CLVA) enables one-shot artistic style transfer using natural language instructions, ranging from visual attributes to emotional effects, effectively preserving content structure and generating high-quality stylizations. The model achieves superior performance in human evaluations and automatic metrics compared to prior methods, while operating orders of magnitude faster and with reduced computational resources.
Key insights
- A patch-wise style discriminator effectively learns nuanced visual semantics from linguistic instructions, enabling precise style transfer at a local level.
- Contrastive Reasoning, through consistent and relative matching losses, significantly enhances the model's ability to maintain content structure and apply consistent stylistic properties across varied inputs.
- The proposed architecture allows for efficient, one-shot stylization by directly learning to map text instructions to stylistic modifications, avoiding iterative optimization common in other text-guided methods.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.