Reproduce “XM-Sparse: High-Throughput Sparse-Attention LLM Serving over Scale-up Memory Pools” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yijun Sun, Xudong Liao, Zhuobin Huang, Kai Chen, Nikita Lazarev, Zhizhen Zhong
XM-Sparse increases long-context decoding throughput by offloading both key/value (KV) history and selection-index (K-index) pages from GPU high-bandwidth memory. It prefetches predictable K-index data and retrieves selected KV tiles while attention runs. On an emulated memory pool, GLM-5.3 at 128K context reaches 1,849.3 tokens/s/GPU, 1.29× a same-platform HiSparse integration, with batch 209 versus 109 requests/GPU.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.