Reproduce “MixServe: An Automatic Distributed Serving System for MoE Models with Hybrid Parallelism Based on Fused Communication Algorithm” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Bowen Zhou, Jinrui Jia, Wenhao He, Yong Zhang, Fang Dong
MixServe is an automatic distributed serving system for Mixture of Experts (MoE) models that leverages intelligent parallelism strategy selection and a fused communication algorithm. The system achieves 1.08x to 3.80x faster Time to First Token and up to 50.3% higher throughput for DeepSeek-R1 and Qwen3 models on Nvidia H20 GPUs and Ascend 910B NPUs.
Key insights
- Optimal parallel strategies for MoE serving are highly dependent on specific model hyperparameters, network topology, and distinct hardware characteristics.
- Rigorous theoretical modeling of communication and computation latency, rather than empirical tuning, is essential for identifying and selecting the most efficient distributed serving configurations.
- Exploiting bandwidth hierarchies through fused, asynchronous communication operations can substantially reduce effective communication latency and improve overall inference performance for MoE models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.