Reproduce “Multi-View Latent MoE: Letting Every Router Shape Expert Computation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zelin Zhao, Junyuan Shang, Tianchi Ma, Shuohuan Wang, Dianhai Yu

The authors introduce Multi-View routing for sparse mixture-of-experts models, combining independently initialized router scores to select experts and using each selected expert’s relative scores to blend its input views. Each expert runs once. On one 24-layer 10B-A1B model trained on 708B tokens, the nine-task mean is 34.25, versus 33.07 for LatentMoE; results cover one seed and include task declines.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.