Reproduce “WeTok: Powerful Discrete Tokenization for High-Fidelity Visual Reconstruction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shaobin Zhuang, Yiwei Guo, Canmiao Fu, Zhipeng Huang, Zeyue Tian, Xiaohui Li, Fangyikang Wang, Ying Zhang and 2 more
WeTok introduces Group-wise Lookup-Free Quantization (GQ) and a Generative Decoder (GD) to advance discrete visual tokenization, achieving state-of-the-art reconstruction fidelity with a zero-shot rFID of 0.12 at 400% compression, outperforming leading continuous tokenizers while maintaining high compression ratios.
Key insights
- By quantizing latent features in groups, memory bottlenecks associated with entropy loss calculations are eliminated, enabling discrete codebooks to scale to arbitrary sizes without prohibitive resource consumption.
- Introducing a noise variable allows the decoder to learn a conditional probability distribution, capturing the full diversity of visual data and reconstructing fine details that deterministic decoders typically average out.
- Discrete tokenizers, when optimized for both codebook scalability and generative decoding, can surpass the reconstruction fidelity of state-of-the-art continuous tokenizers while retaining superior compression efficiency.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.