Reproduce “ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Haoran You, Yipin Guo, Yichao Fu, Wei Zhou, Huihong Shi, Xiaofan Zhang, Souvik Kundu, Amir Yazdanbakhsh and 1 more

A ShiftAddLLM framework accelerates pretrained Large Language Models by reparameterizing dense multiplications into hardware-efficient shift-and-add operations post-training. The method achieved over 80% memory and energy reduction while maintaining competitive accuracy at low bit-widths, outperforming existing baselines on perplexity and downstream tasks.

Key insights

  • Pretrained LLMs can be effectively reparameterized into multiplication-less shift-and-add operations in a post-training manner, overcoming the need for expensive retraining or extensive fine-tuning.
  • Mitigating accuracy loss at low bit-widths is crucial and can be achieved through a multi-objective optimization that adaptively considers both weight quantization and output activation errors.
  • Different architectural layers and blocks within an LLM exhibit varying sensitivities to reparameterization, necessitating an automated, non-uniform bit allocation for optimal accuracy-efficiency trade-offs.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.