Reproduce “ViTok-v2: Scaling Native Resolution Auto-Encoders to 5 Billion Parameters” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Philippe Hansen-Estruch, Jiahui Chen, Vivek Ramanujan, Orr Zohar, Yan Ping, Animesh Sinha, Markos Georgopoulos, Edgar Schoenfeld and 4 more
ViTok-v2 introduces a Vision Transformer-based autoencoder that scales to 5 billion parameters, supporting native resolution processing up to 8K imagery. It achieves state-of-the-art reconstruction quality and stable training through a novel DINOv3 perceptual loss, enabling efficient high-resolution generative models where previous architectures struggled.
Key insights
- Scaling the decoder's parameter count substantially improves reconstruction metrics, particularly at higher compression ratios, demonstrating the importance of decoder capacity for challenging compression tasks.
- The DINOv3 perceptual tile loss provides superior perceptual quality and training stability compared to LPIPS and adversarial losses, effectively capturing fine-grained spatial and semantic information.
- For high-channel latents, simpler latent regularization methods like LayerNorm are sufficient, yielding generation quality comparable to more complex stochastic approaches like KL divergence.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.