Reproduce “Smarter and Cheaper at Once: Byte-Exact KV-Cache Grafting Turns a Frozen Small Model into a Verified-Knowledge Flywheel” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sietse Schelpe
Researchers from Corbenic AI introduced byte-exact KV-state grafting, a method to persistently store and restore internal KV states, allowing frozen, smaller language models to enhance capabilities and reduce costs. A Gemma-4-12B model achieved 93.3% on AIME 2025, surpassing the 31B model's 89.2% anchor, while reducing tokens by 6,574x and energy by up to 8,700x for recurring problems.
Key insights
- Byte-exact KV-state grafting provides a durable, precise mechanism for injecting verified knowledge into frozen models, eliminating recurring computation for learned contexts.
- The numerical exactness for rotary position embeddings is strictly limited to "own-position graft" due to the base model's inherent floating-point sensitivity, preventing byte-exact relocation.
- Converting recurring computational costs into a one-time investment via verified KV-state storage enables a form of "inference-time learning" that makes frozen models both more capable and cost-efficient.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.