Reproduce “Vector Quantized Diffusion Model for Text-to-Image Synthesis” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang, Dongdong Chen, Lu Yuan, Baining Guo

A Vector Quantized Diffusion Model (VQ-Diffusion) for text-to-image synthesis integrates VQ-VAE with a conditional discrete diffusion process to address limitations of prior autoregressive models. The approach achieved an FID score of 13.86 on MSCOCO and enabled inference up to 15 times faster than autoregressive counterparts.

Key insights

  • Discrete diffusion models, when combined with a VQ-VAE latent space, offer a powerful and efficient alternative to autoregressive and GAN-based approaches for text-to-image synthesis.
  • The proposed 'mask-and-replace' discrete diffusion strategy effectively eliminates unidirectional bias and mitigates error accumulation inherent in autoregressive models, leading to more coherent and higher-quality image generation.
  • A fast inference strategy, leveraging the ability to predict the noiseless token distribution, allows for significant speed-ups (up to 15x) without substantial quality degradation, addressing a major practical limitation of generative models.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.