Reproduce “FreeAct: Freeing Activations for LLM Quantization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiaohao Liu, Xiaobo Xia, Manyi Zhang, Ji-Fu Li, Xianzhi Yu, Fei Shen, Xiu Su, See-Kiong Ng and 1 more
Researchers developed FreeAct, a post-training quantization method for Large Language Models that relaxes the static one-to-one transformation constraint to accommodate dynamic activation patterns. This approach demonstrates superior accuracy retention for diffusion and multimodal LLMs at ultra-low bit-widths (W4A4), recovering performance comparable to full-precision models in some cases and outperforming baselines by up to 5.3%.
Key insights
- The rank-deficient nature of LLM activations allows for a broader solution space beyond simple inverse matrices, enabling the decoupling of activation transformations from weights.
- Allowing dynamic, input-characteristic-dependent transformations for activations, while keeping weight transformations static, is crucial for improving quantization quality in LLMs with diverse input types.
- A carefully designed framework using token indexing and shared/unique transformation components can mathematically guarantee equivalence during linear operations, even with distinct activation transformations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.