Reproduce “Lachesis: Lifetime-Aware KV Cache Placement for Agent Serving across HBM and High-Bandwidth Flash” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jaehoon Yang, Jeongmin Lee, Haneul Park, Seung Yul Lee, Nam Sung Kim, Jae W. Lee
Lachesis places agent-session KV-cache segments in HBM or high-bandwidth flash (HBF) according to their expected lifetime, freeing short-lived blocks at harness-defined boundaries and ranking workers when their completion times are uncertain. In trace-driven simulation under a tight service-level objective, it extends modeled HBF lifetime 1.50–2.95× over HBM-first across four configurations on both models; these are projections, not physical-device measurements.
Key insights
- In the temporal traces, reasoning-stripped context lasts 7.6–17.2× longer by median lifetime than reasoning-laden context, which represents 66–83% of a turn’s KV writes.
- Lead-agent context lasts longer than worker or reviewer context, and workers vary substantially in lifetime; spawn-time ranking reaches Kendall’s τ of 0.53.
- Read-to-write behavior alone can be a poor proxy for lifetime: lead-agent context may remain unread while sub-agents work, yet have a read-to-write ratio similar to a worker’s.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.