Reproduce “TripleKV: Predict, Prefetch, Protect -- KV Cache Management for AI Agents” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Shashwat Jaiswal, Chloe Alverti, Mengmei Ye, Chen Wang, Hubertus Franke, Fan Lai, Tianyin Xu, Josep Torrellas

The authors introduce TripleKV, a system that prefetches an agent’s known KV-cache blocks from SSD to host DRAM while a tool call runs, then loads them into GPU memory when inference resumes. It predicts timing and protects prefetched blocks from eviction. Across four benchmarks, it lowers critical-path SSD reads to under 10% of requests and cuts GAIA P95 TTFT by up to 49% versus vLLM with L3.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.