Reproduce “KVStream: Adaptive Parallel Storage Management for KV Caching in Large-Scale LLM Serving” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Avinash Maurya, Jaiaid Mobin, Jie Ye, M. Mustafa Rafique, Franck Cappello, Bogdan Nicolae
KVStream retrieves reusable prompt cache chunks from storage. It replicates chunks across node-local SSDs and a shared file system, then lets each tier claim more reads as batches finish rather than using a fixed split. On a 32B SWE-bench trace, mean TTFT advantage over LMCache reaches 6.07× at arrival CV 4; it is slightly slower when host memory holds the working set.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.