Reproduce “SALS: Sparse Attention in Latent Space for KV cache Compression” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Junlin Mu, Hantao Huang, Jihang Zhang, Minghui Yu, Tao Wang, Yidong Li
SALS reduces long-context key–value cache access by using a compact pre-RoPE representation to rank tokens, then reconstructing and applying RoPE only to selected keys for final attention. On LLaMA2-7B-chat LongBench, SALS-12.5% reports an average score of 31.97 and normalized memory access of 0.06, close to Loki’s score of 31.95 but below the full-attention baseline’s 32.65.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.