Reproduce “Efficient MoE Inference with Fine-Grained Scheduling of Disaggregated Expert Parallelism” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xinglin Pan, Shaohuai Shi, Wenxiang Lin, Yuxin Wang, Zhenheng Tang, Wei Wang, Xiaowen Chu
The FinDEP framework introduces a fine-grained scheduling algorithm for Disaggregated Expert Parallelism (DEP) in Mixture-of-Experts (MoE) model inference. It optimizes resource utilization and task overlap by partitioning computation and communication, achieving up to 1.61x throughput improvement over state-of-the-art methods on various GPU systems for DeepSeek-V2 and Qwen3-MoE models.
Key insights
- Fine-grained partitioning of tasks is crucial for effectively overlapping computation and communication in MoE inference, especially for models with shared experts.
- Accurate performance models for GPU operations and network communication are essential for formulating effective optimization problems in distributed AI systems.
- Exploiting mathematical properties like monotonicity and convexity within the optimization objective allows for the efficient exploration of large parameter spaces, enabling dynamic adaptation to varying workloads.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.