Reproduce “Accelerating Mixture of Block Attention via Dynamic Routing, KV-Centric Execution and Runtime Pipelining” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yingqi Shi, Junru Lu, Yun Liu, Di Yin, Xing Sun

LightningMoBA speeds long-context prefill by retaining KV blocks within a query-relative score threshold, then grouping work by KV block to reuse data and schedule GPU tasks. Against FlashMoBA at matched average Top-16 retention, it reduced generation-loop time across five checkpoints, with 1.20–1.60× speedups, while mean RULER rose from 74.42 to 75.28; scores fell for two models and all five at 64K.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.