Reproduce “OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimization” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xiang Meng, Shibal Ibrahim, Kayhan Behdin, Hussein Hazimeh, Natalia Ponomareva, Rahul Mazumder

OSSCAR (One-Shot Structured Compression AlgoRithm) is a framework for structured pruning of large neural networks, including vision and language models up to 30 billion parameters, without requiring retraining. The framework achieves dramatically better performance and improved efficiency over existing one-shot pruning methods, such as a 125x lower perplexity for OPT-2.7B at a 2x speedup compared to previous state-of-the-art.

Key insights

  • Structured pruning can be effectively framed as a combinatorial optimization problem, minimizing layer-wise reconstruction error.
  • The quadratic objective in this problem possesses a specific structure that can be exploited for significant computational complexity reduction using low-rank matrix updates.
  • Efficiently solving this optimization allows for one-shot structured pruning of multi-billion parameter models (like LLMs) without retraining, while substantially preserving performance.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.