Reproduce “Birkhoff Decompositions and Photonic Interconnects Wait! Don't Forget the Compute!” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Eliezer Amponsah, Vamsi Addanki

This work from Purdue University analyzes how different circuit scheduling methods impact end-to-end performance for Mixture-of-Experts (MoE) models using reconfigurable photonic interconnects. It reveals that communication-optimized decompositions often fragment expert computation, leading to significant inefficiencies, and demonstrates that a greedy max-weight decomposition can substantially reduce execution makespan by preserving larger token batches, especially for large workloads.

Key insights

  • Communication-optimal decompositions are not necessarily execution-optimal for MoE workloads, as they can severely reduce compute efficiency due to fragmented expert batches.
  • Expert computation exhibits non-linear efficiency with batch size; small batches incur disproportionately high fixed overheads, often negating communication benefits.
  • Effective MoE scheduling requires prioritizing large execution batches to amortize compute overheads and provide sufficient windows for hiding communication and reconfiguration latency.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.