Reproduce “When Reasoning Meets Compression: Understanding the Effects of LLMs Compression on Large Reasoning Models” on OpenResearch

About this paper

Nan Zhang, Eugene Kwek, Yusen Zhang, Ngoc-Hieu Nguyen, Prasenjit Mitra, Rui Zhang

Research identifies that the `mlp.up_proj` matrix in the final layer is a critical component for reasoning in Large Reasoning Models (LRMs) and that current quantization methods frequently over-compress this and other essential modules. Protecting approximately 2% of the total weights in these final-layer MLP modules during 3-bit quantization resulted in an average accuracy improvement of 6.57% and up to a 23.17% gain over standard 3-bit baselines.

Key insights

• The `mlp.up_proj` matrix in the final layer consistently emerges as one of the most important model components for various reasoning behaviors across R1 distilled Llama and Qwen models.
• State-of-the-art quantization methods (AWQ, GPTQ) tend to over-compress critical modules, including MLP gate projections in middle/early layers and most linear modules in the final layer.
• Model parameter count more heavily impacts knowledge memorization than reasoning capabilities, with pruning disproportionately harming knowledge retention compared to quantization.

Summary by alphaXiv.