Reproduce “AgentKV: Phase-Aware KV Eviction for Agentic LLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Taowen Tony Liu, Jeffrey T. H. Wong, Can Xiao, Bowen Yang, Hao Mark Chen, Yiren Zhao
AgentKV treats an agentic language model’s KV-cache demand as phase-dependent rather than adequately represented by the latest queries alone. By retaining representative queries from reasoning, tool-call, tool-response, and other context phases, and combining them with physical cache compaction, it improves task quality over recency-oriented eviction while delivering up to 1.80× higher decoding throughput.
Key insights
- Query representations occupy distinct phase-specific subspaces: measured cross-phase principal angles are generally larger than within-phase angles, indicating that one recent-query distribution can miss future demand.
- Phase-aware selection especially improves preservation of earlier tool information; at a 512-token budget, tool utility coverage rises from 0.042 with recency representatives to 0.175 with multi-phase representatives.
- The method remains reasonably tolerant to imperfect phase labels, but splitting the fixed representative budget across too many phases or using very small cache budgets can reduce its advantage.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.