Reproduce “Kameleo: In-Situ Activation-Precision Selection for Bit-Serial LLM Accelerators” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Robin Geens, Thomas Debelle, Marian Verhelst, Chao Fang
Kameleo selects activation mantissa lengths for weight-only language models during prompt prefill. It uses a bit-serial accelerator’s partial results to compare reduced-precision matrix multiplications with full-precision results, then chooses lengths in two passes on the current workload. Across four models and seven datasets, it keeps perplexity degradation within 1% while reducing matrix multiplication unit cycles 2.11–2.34×; these are prefill cycles, not end-to-end latency, on tested data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.