Reproduce “DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
DeepSeek-AI, Anyi Xu, B. Li, Bangcai Lin, Bing Xue, BingCheng Xian, Bingzheng Xu, Bochao Wu and 584 more
DeepSeek-V4.1-Flash compresses persistent and runtime key-value caches for long-horizon agents by sharing cache representations across layers, storing main cache in FP4, and rebuilding sliding-window states with bounded replay. It reports 890 bytes per token for global cache and one-eighth V4-Flash persistent storage. On DeepSWE v1.1, it scores 74.2 versus 54.4, but replay is approximate and untested boundaries could degrade capability.
Key insights
- KV-cache efficiency is addressed across value precision, sequence sparsity, layer sharing, and selective recomputation rather than through a single compression method.
- Cross-layer reuse and bounded local-window replay make single-token decode FLOPs nearly constant as context grows from 4K to one million tokens, although sparse retrieval and approximate replay can introduce failure cases.
- Reasoning effort can be varied at inference time: increasing the setting from 25 to 100 raises average performance across eight reasoning benchmarks from 67.1% to 76.3%, at roughly 2.5 times the output-token cost.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.