Reproduce “PagedMoE: Cross-Precision Expert Caching for Token-Level Dynamic Precision in SSD-Resident MoE Inference” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yutong Lin, Yingdi Shan, Shaoyuan Chen, Yuanyong Chen, Yongwei Wu, Mingxing Zhang
PagedMoE improves batch-size-one mixture-of-experts inference by storing expert weights as shared precision-depth prefixes that can be extended from SSD and reused in DRAM. It selects bit-depths using routing weights and offline error profiles, then executes compact prefixes on CPUs. On Qwen3.5-397B-A17B, throughput reaches 8.89 tokens/s versus 6.67 with independent precision objects at 24 GiB, a decoding result.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.