Reproduce “MoEntwine: Unleashing the Potential of Wafer-scale Chips for Large-scale Expert Parallel Inference” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xinru Tang, Jingxiang Hou, Dingcheng Jiang, Taiquan Wei, Jiaxin Liu, Jinyi Deng, Huizheng Wang, Qize Yang and 4 more
MoEntwine co-designs attention placement and expert-parallel MoE inference on wafer-scale systems to reduce token traffic and schedule expert migration over spare mesh links. ER-Mapping trades longer attention all-reduce for shorter MoE all-to-all, while NI-Balancer stages weight moves during idle-link opportunities. In profile-and-simulate evaluations, a 256-device system reports 39% higher average per-device MoE performance than NVL72 across Qwen3 and DeepSeek-V3.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.