Reproduce “Adaptive Pruning for Large Language Models with Structural Importance Awareness” on OpenResearch

About this paper

Haotian Zheng, Jinke Ren, Yushan Sun, Ruichen Zhang, Wenbo Zhang, Zhen Li, Dusit Niyato, Shuguang Cui and 1 more

SAAP, a structured pruning framework for large language models, employs adaptive importance assessment and group-wise fine-tuning to significantly reduce computational and memory costs. It achieves up to 65% faster token generation and substantially lowers memory footprint on various LLMs like LLaMA and Vicuna, while maintaining or improving performance across diverse benchmarks.

Key insights

• Fusing coarse-grained and fine-grained importance scores adaptively, weighted by their observation noise, provides a more robust and accurate basis for pruning decisions across LLM architectures.
• Pruning structures based on their dynamic importance volatility and stability, rather than static magnitude, allows for more effective layer-wise compression while preserving model quality.
• Integrating quantization directly into a group-wise low-rank fine-tuning approach significantly reduces memory overhead and improves deployment efficiency by ensuring quantized weights throughout the recovery stage.

Summary by alphaXiv.