Reproduce “2DQuant: Low-bit Post-Training Quantization for Image Super-Resolution” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kai Liu, Haotong Qin, Yong Guo, Xin Yuan, Linghe Kong, Guihai Chen, Yulun Zhang
2DQuant presents a two-stage post-training quantization method specifically for Transformer-based image super-resolution models, addressing their unique architectural and distribution challenges. This approach enables significant model compression (up to 3.60x) and inference speedup (up to 5.08x) while maintaining high image quality, exemplified by a 4-bit quantized SwinIR achieving 26.83 dB PSNR on Urban100 with only marginal degradation from the 26.94 dB full-precision baseline.
Key insights
- Transformer-based SR models exhibit diverse and complex weight and activation distributions (symmetric, asymmetric, long-tail) that make traditional one-size-fits-all quantization ineffective.
- A coarse-to-fine two-stage quantization approach, combining adaptive bound initialization with distillation-based refinement, effectively minimizes performance degradation in low-bit Transformer models.
- Feature-level distillation is crucial for preventing error accumulation and maintaining high-quality results in deep quantized networks, specifically aligning intermediate representations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.