Reproduce “Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse” on OpenResearch

About this paper

Taekyung Heo, Rasoul Shafipour, Ritchie Zhao, Maximilian Golub, Mohammad Mahdi Kamani, Ritika Borkar, Makesh Tarun Chandran, Pantea Zardoshti and 1 more

The authors present a linear ridge mapper that transfers an attention cache between matched-KV language models, avoiding the receiving model’s prompt prefill. It selects and maps source-layer keys and values, with position handling for keys. In a Qwen3 14B-to-32B test at 32K tokens, transfer took 277.6 ms versus 6,975.3 ms for re-prefill; this measures prefill latency, not end-to-end conversation time.

Key insights

• Substantial linear structure exists in cross-model KV cache relationships, especially for RoPE-stripped keys, enabling efficient gradient-free mapping between related LLMs.
• Cross-layer information distribution is crucial, with multiple source layers contributing complementary information necessary for accurately predicting a target layer's KV cache.
• Traditional reconstruction metrics like R^2 are unreliable predictors of downstream task accuracy; attention-output cosine similarity and the distribution of errors in attention-sensitive subspaces better correlate with transfer quality.

Summary by alphaXiv.