Reproduce “Recompute or Restore? Host-Tier KV for Multi-Model Agent Workflows on a Static-Graph NPU Serving Stack” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sungwon Woo, Seyeon Kim
The authors measure a host-memory tier for reusing evicted key-value (KV) prefixes on a static-graph accelerator, with a scheduler-aware model whose predictions were frozen before each test cohort. Transfers share the planner’s engine thread with inference. In one KV32 mixed-burst overload run, the tier raised SLO attainment from 9.4% to 99.8%; at common load, it increased fork P99 latency 15.6–17.5%.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.