Reproduce “Eliminating VAE for Fast and High-Resolution Generative Detail Restoration” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yan Wang, Shijie Zhao, Junlin Li, Li Zhang
GenDR-Pix, developed by MMLab, ByteDance, introduces a diffusion-based super-resolution model that eliminates the Variational Autoencoder (VAE). This architectural change results in approximately 2.8x acceleration and over 60% GPU memory savings, allowing for 8K image restoration without tiling while maintaining restoration quality.
Key insights
- The VAE is identified as the primary bottleneck for both efficiency and fidelity in one-step diffusion SR models, and its complete removal is achievable with substantial benefits.
- Directly replacing the VAE with pixel(un)shuffle operations, especially at large scales, requires specific techniques like MFS loss to address artifacts in the frequency domain.
- Careful adversarial distillation, combined with data augmentation like RandPad, is essential to stabilize training and maintain generative quality when transitioning to a VAE-free, pixel-space model.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.