Reproduce “RISA: Recoverable Sparse Attention for Block-Diffusion Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xinrui Zhong, Fanjiang Ye, Yuke Wang

RISA reduces repeated historical reads in block-wise diffusion models by preserving exact key-value history in priority and recovery tiers, selecting one bounded page plan per block, and reusing its addresses while recomputing attention with fresh queries. On DreamReasoner-8B at a 32K prefix, a complete 512-token request is 1.36× faster than dense, with the same denoising steps.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.