Reproduce “Cache Engineering for Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tianyu Fu, Ruoxi Wang, Zihan Min, Zhongqi Ren, Zhihao Li, Tengxuan Liu, Shengxu Mu, Hanling Zhang and 1 more
AKV lets agents manage active key–value cache states without changing message histories. It drops selected states from future attention and reuses compatible retained states, avoiding some recomputation. Across three models and four benchmarks, it reports a 1.7-point average pass-rate gain and 27.7% lower estimated API-equivalent cost than threshold-triggered summarization, with results limited to evaluated settings.
Key insights
- A cache state is determined not only by its retained tokens and positions, but also by which earlier entries were visible when those tokens were computed; token-prefix matching alone is therefore insufficient for safe reuse.
- Older tool-response cache can be dropped while retaining reasoning cache: reported ablations favored this strategy over dropping reasoning or both categories.
- Dropping entries does not reset position indices. Repositioning addresses position limits, and on a reported BrowseComp-Plus subset raised the tool-drop policy score from 55% to 62%.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.