Reproduce “Joint Subspace Sparse Attention for Efficient Long-Context Decoding” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jiwon Song, Yoongon Kim, Jae-Joon Kim

JSSA reduces long-context decoding attention reads using a shared low-dimensional query–key scoring space and a gather-free kernel that attends directly to selected cache entries. It scores every candidate and retains the original key–value cache, adding projected scoring state. On GLM-4.7-Flash, it reaches up to 6.43× throughput over full-KV attention at 128K context on one B200, with a 2,048-token budget; gains vary by workload.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.