Reproduce “vExpert: Virtualizing Expert Storage for Adaptive Load Balancing in Distributed MoE Inference” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wenxun Wang, Xiuhong Li, Yida Wang, Chen Tang, Zongle Huang, Ke Hong, Yu Wang, Yongpan Liu
vExpert adapts distributed Mixture-of-Experts decoding by keeping expert weights in host DRAM and streaming selected copies into GPU slots when balancing gains justify transfer costs. On 671B DeepSeek-V3.1 at batch size 256, it reaches a load-balance ratio of 0.76 versus 0.49 for the strongest stated baseline, and 28.4% higher throughput than vanilla SGLang, on the tested setup.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.