Reproduce “LoopVQ: Sub 1-bit KV Cache Quantization for Looped Transformers” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Minsoo Cheong, Sungjoo Yoo

LoopVQ compresses looped language models’ key-value caches by coding channels across four loop steps together and fine-tuning codebooks to match full-precision outputs. This exploits redundancy between steps, while output-aware tuning accounts for unequal effects on model outputs. At 0.5 bits per element, Ouro-2.6B scores 77.69 average accuracy, versus 37.67 for CQ and 81.46 for BF16, across four benchmarks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.