Reproduce “CogView3: Finer and Faster Text-to-Image Generation via Relay Diffusion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wendi Zheng, Jiayan Teng, Zhuoyi Yang, Weihan Wang, Jidong Chen, Xiaotao Gu, Yuxiao Dong, Ming Ding and 1 more
The CogView3 model from Tsinghua University and Zhipu AI presents a relay diffusion framework for text-to-image generation, achieving finer detail and up to 10x faster inference at high resolutions compared to current state-of-the-art models like SDXL. This approach leverages a novel cascaded design and advanced text preprocessing to enhance both image quality and generation speed.
Key insights
- Relay diffusion effectively corrects unsatisfactory artifacts from a base generation by allowing the super-resolution stage to re-diffuse from a noised version of the low-resolution input.
- Aligning training data captions with expanded user prompts significantly enhances the model's ability to interpret and follow complex instructions.
- The relay diffusion architecture facilitates efficient progressive distillation, enabling drastic reductions in inference steps (e.g., 4+1 steps) with minimal impact on generated image quality.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.