Reproduce “BellowsKV: One Latent for Many Layers, Expanded on Demand” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiamu Zhang, Hanjie Chen
BellowsKV compresses long-context cache memory by storing a shared hidden-state summary for adjacent layers, then reconstructing each layer’s own keys and values. It keeps the latent dimension full and quantizes it; sink positions, recent prompt positions, and generated tokens remain exact. On Llama-3.1-8B-Instruct, it achieves 98.0% average task-score retention at 13.84× compression, versus xKV’s 83.5% at 16×, so the ratios differ.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.