Reproduce “A Style is Worth One Code: Unlocking Code-to-Style Image Generation with Discrete Style Space” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Huijie Liu, Shuhao Cui, Haoxiang Cao, Shuai Ma, Kai Wu, Guoliang Kang
CoTyle, an open-source framework developed by researchers at Beihang University and Kuaishou Technology, introduces code-to-style image generation using diffusion models. It achieves high stylistic consistency and creativity by converting a numerical code into a robust style controller, outperforming Midjourney in consistency (0.6007 vs. 0.4734) and improving aesthetics.
Key insights
- A discrete style codebook, trained with a multi-component loss including contrastive and reconstruction terms, can effectively extract and distinguish nuanced visual styles from image content.
- Injecting learned style embeddings through the textual branch of a Diffusion Transformer, leveraging a Vision Language Model, proves more effective for semantic style transfer and consistency than visual conditioning.
- An autoregressive model, guided by a simple numerical code and a high-frequency suppression strategy, can navigate a discrete style space to generate diverse and novel stylistic patterns.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.