Reproduce “Empowering Hybrid Attention Models on NPUs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yinyuan Zhang, Daliang Xu, Xiaolong Huang, Wangsong Yin, Yun Ma, Mengwei Xu, Gang Huang
HA-NPU reorganizes linear-attention prefill on Ascend NPUs without changing the attention algorithm. It maps a head’s dependent work to one core, reorders operations to shorten intermediate lifetimes, and plans tensor layouts to reduce data movement. On Ascend 310P, its cumulative optimizations lower GDN and KDA kernel latency; end-to-end request latency improves by up to 2.03× versus vLLM-Ascend, while decoding remains.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.