Reproduce “Adaptive Model Compression (AMC): Saliency-Driven Resource Allocation for Ultra-Low-Power Transformer Inference” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiayin Hu, Kai Yuan, Vanessa Hu, Xuetao Yin, Jianhua Li, Sean Suchter
Researchers at Apple USA introduced Adaptive Model Compression (AMC), a hardware-software co-design framework that dynamically adjusts computational resources for transformer inference based on token saliency. Implemented on a 45nm CMOS technology, AMC reduced system energy consumption by 59.2% and increased throughput by 2.24x with a marginal 3.6% accuracy trade-off, enabling efficient deployment on edge devices.
Key insights
- Activation magnitude, specifically the L1-norm of activation vectors, serves as an effective, non-intrusive heuristic for identifying token importance, enabling selective resource allocation without significant accuracy degradation.
- A tight hardware-software co-design, where software dynamically informs hardware gating and precision adjustments, significantly outperforms purely software-based or static hardware compression methods in energy efficiency and throughput.
- Offline dimension structural alignment (via SVD/PCA) and sequence-adaptive threshold adjustment allow the framework to preserve high-variance features during rank reduction and dynamically adapt to varying linguistic complexities.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.