Reproduce “LowRAM: Low-Rank Keys and Amortised Values for Composable KV Cache Compression at Test-Time” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Teresa Delgado, Hugo Hazard, Martin Benfeghoul, Jun Wang, Zafeirios Fountas
The authors introduce LowRAM to compress a frozen model’s long-context cache at test time. It stores keys in low-rank form, predicts values from keys, and retains exact corrections for high-error tokens. On Mistral-7B LongBench prompts over 10K tokens, LowRAM with eviction and quantisation reaches 26.9× cache compression at 94.5% recovered performance, the maximum compression retaining at least 90% on this subset.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.