Reproduce “RIFT: KV Cache Compression for Draft Models with Complementary Compressibility Training” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dor Tsur, Maor Ashkenazi, Maximilian Golub, Peter Hicks, Yaniv Galron, Talor Abramovich, Yoav Miron, Tiyasa Mitra and 2 more
RIFT combines recent tokens with selected prompt context and refreshes selections as generation proceeds. Compression-aware training prepares the draft for these sparse cache views. For Qwen3.5-4B drafting for Qwen3.5-122B, the combined method retained over 98% of full-cache average acceptance length on 32K-token inputs and 4K-token outputs with an 8K-token active-cache budget; this is not a measured end-to-end speedup.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.