Reproduce “FlagRot: Selective Rotation for Low-Bit KV Caches” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kunming Shao, Xijie Huang, Jierun Chen, Jiaqi Shao, Yanli Wang, Jiangnan Yu, Qiwei Li, Wei Zhang and 2 more
FlagRot compresses key–value cache memory for long-context inference by selecting, for each key piece, either an unrotated or rotated representation. It estimates candidates’ attention-logit error using calibration queries, rather than relying only on reconstruction error. At equal 2.5-bit storage, FlagRot-KIVI lowers mean next-token KL from 18.7 to 12.4 across 12 models; task gains are not universal.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.