Reproduce “Compositional Performance Modeling of LLM Inference Clusters with Hierarchical KV Caching” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ruiyang Ma, Sheng Qi, DaiKang Kuang, Teng Ma, Guojie Luo
The authors present an event-driven simulator that combines locally calibrated compute and transfer costs with KV residency, capacity reservations, resource queues, and completion-dependent availability. Reusing profiles calibrated on one 1P–1D setup, it predicts end-to-end latency across SGLang Qwen3-8B and Qwen3-32B clusters with mean MAPE of 1.34% for QA and 6.36% for coding. Results apply to tested compatible configurations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.