Reproduce “SlimLLM: Accurate Structured Pruning for Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jialong Guo, Xinghao Chen, Yehui Tang, Yunhe Wang
Huawei Noah's Ark Lab developed SlimLLM, a structured pruning method that compresses large language models while maintaining high accuracy. The approach achieved 98.7% performance retention on LLaMA-7B after 20% pruning and reduced LLaMA-7B prefill latency by 65% and decoding latency by 28.5% at 50% pruning on NVIDIA V100 GPUs.
Key insights
- Holistic importance criteria, which consider interdependencies and directional information, lead to more accurate pruning decisions than magnitude-only approaches for both attention heads and FFN channels.
- A simple yet effective linear regression strategy can rapidly and efficiently recover significant performance loss after pruning, reducing the need for extensive fine-tuning.
- LLM layers exhibit varying levels of redundancy, making adaptive, non-uniform pruning ratios crucial for optimal compression and performance retention, with deeper layers often tolerating higher pruning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.