Reproduce “Democratizing MoE inference on commodity GPUs with CoMoE” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ruwen Fan, Yuezhi Zu, Minhui Xie, Junru Li, Qingda Hu, Xinjun, Yang, Jiwu Shu and 1 more
CoMoE redesigns multi-GPU mixture-of-experts inference communication by using host memory as a dispatch hub and staging point for combining outputs as tokens become ready. It reduces source-side sends and batch-wide waits, while shifting demand to receiver reads and host bandwidth. On 8 RTX 5090 GPUs, GPT-OSS-20B throughput reaches 7,304.18 output tokens/s, 1.46× the SGLang NCCL baseline, but remains below A800 DeepEP.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.