Reproduce “Pareto-Optimal Quantized ResNet Is Mostly 4-bit” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
AmirAli Abdolrashidi, Lisa Wang, Shivani Agrawal, Jonathan Malmaud, Oleg Rybakov, Chas Leichner, Lukasz Lew
Researchers from Google Research and UC Riverside investigated the efficiency-quality tradeoff of quantized ResNet models by varying network size and quantization bit-width. They demonstrated that a "mostly 4-bit" quantization scheme consistently achieves Pareto-optimal performance, often yielding higher accuracy than bfloat16 and 8-bit models while significantly reducing compute and memory costs.
Key insights
- Quantization, especially at 4-bit precision (with 8-bit for initial/final layers), is Pareto-optimal, consistently dominating higher-precision models across various network scales in compute cost-accuracy tradeoffs.
- Quantization exhibits a regularizing effect, leading to improved generalization (smaller generalization gaps) and often higher Top-1 accuracy compared to unquantized bfloat16 baselines.
- The advantages of low-bit quantization become even more pronounced under an energy-motivated quadratic cost model, providing strong motivation for future hardware acceleration.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.