Reproduce “RailWave: Adaptive Spatial and Temporal Scheduling for Expert-Parallel Communication” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Chutian Wang, Wenhao He, Jingmin Zhu, Qingyu Yin, Heng Xu, Xiuyu Li

RailWave reshapes dispatch–combine communication in mixture-of-experts models without changing routing or logical demand. It spreads transfers across eligible source Rails and schedules node-pair exchanges in waves to reduce Rail imbalance and receiver contention. A selector chooses among these mechanisms. Across three replay patterns, P50 communication speedups reach 5.84× on H800 and 4.36× on H20. These are communication measurements, not end-to-end training speedups.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.