Reproduce “ActKV: Efficient LLM Agents through Action-Guided KV Cache Management” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zihan Wang, Cheng Tang, Lei Gong, Chao Wang, Wenqi Lou, Teng Wang, Xuehai Zhou
ActKV manages an agent’s growing key-value cache by retaining entries that receive attention during action generation and adjusting the cache budget as confidence changes. In controlled tests, this selection outperformed observation- and reasoning-based selection at equal retention. On ALFWorld and WebShop, it reached 98.53% of FullKV accuracy with 25.98% of peak KV memory; these are averages for those tasks.
Key insights
- For agent workloads, attention associated with generated actions can provide a more task-aligned signal for cache retention than treating all reasoning and observation tokens alike.
- Useful context may become relevant again after periods of low attention, so decay-based retention can capture intermittent value while eventually removing stale entries.
- Compression quality depends on systems integration: page-aware compaction is needed for token-level eviction to translate into reusable memory and serving capacity.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.