Reproduce “VFold: Symmetry-Aware Cross-Layer Value Cache Compression” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Neha Verma, Sungwon Kim, Kenton Murray, Kevin Duh
VFOLD aligns neighboring layers’ value vectors offline, then shares their key-value (KV) cache—the stored past-token vectors—only after each layer uses current values in attention. At half the value-cache size, or 25% less total KV memory, it retains at least 98.3% of full-cache performance across three models and two benchmarks, limited to tested settings.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.