Reproduce “Bridging Continuous and Discrete Tokens for Autoregressive Visual Generation” on OpenResearch

About this paper

Yuqing Wang, Zhijie Lin, Yao Teng, Yuanzhi Zhu, Shuhuai Ren, Jiashi Feng, Xihui Liu

TokenBridge, a collaboration between the University of Hong Kong, ByteDance Seed, École Polytechnique, and Peking University, proposes a method for autoregressive visual generation that discretizes pre-trained continuous VAE latent features after training. It achieves an ImageNet 256x256 FID of 1.55, matching or surpassing complex diffusion-based token prediction methods while demonstrating a 5.94x faster inference speed.

Key insights

• High-fidelity visual generation from continuous VAE latents can be achieved with simple categorical prediction (using cross-entropy loss) through judicious post-training discretization, challenging the necessity of complex continuous distribution models.
• Modeling inter-channel dependencies using a dimension-wise autoregressive approach is crucial for high-quality visual generation, as simple parallel prediction leads to significantly degraded results.
• The inherent properties of VAE latent spaces (bounded range, near-Gaussian distribution) enable robust and effective training-free dimension-wise quantization, which generalizes across various autoencoder architectures.

Summary by alphaXiv.