Reproduce “Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xingyu Zhu, Ziheng Cheng, Ang Lv, Jing Liu, Lexing Ying, Yiyuan Ma, Xin Dong
The paper shows that chunked KV-cache compression can make retrieval depend on a token’s position, or phase, within the compression stride. In DeepSeek-V4-Flash-Base, a controlled 128K-context needle task with 16,000 key–value pairs produces a 40.2-percentage-point best-to-worst phase gap. The code example repeats every four tokens. The finding concerns retrieval, not measured speedups or all shifted text.
Key insights
- Retrieval accuracy can vary periodically with source phase; in tested DeepSeek models, best-to-worst gaps reached 40.2 percentage points before post-training and remained as large as 19.1 points afterward.
- All tested chunk-compressed models showed phase variation, with gaps reaching 78 percentage points in the reported sweep; full-attention baselines had gaps no larger than 6.1 points.
- Both boundary effects and within-window encoding or retrieval differences matter. In controlled models, learned gate preferences and phase-dependent component contributions explain some patterns, but not all.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.