Reproduce “The Expert Strikes Back: Interpreting Mixture-of-Experts Language Models at Expert Level” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jeremy Herbst, Stefan Wermter, Jae Hee Lee
The authors test whether sparse mixture-of-experts models are more interpretable than dense models, using k-sparse probes to predict 58 concepts across 12 models. MoE probes outperform dense counterparts, with the largest gap at one neuron. OLMoE-1B-7B also outperforms OLMo-7B despite the latter having seven times more active parameters. This supports an association, not proof that routing sparsity causes monosemanticity.
Key insights
- Mixture-of-Experts (MoE) architectures exhibit reduced polysemanticity at the neuron level compared to dense networks, primarily driven by sparse routing mechanisms.
- Entire MoE experts function as coherent, interpretable computational units, enabling a scalable approach to understanding model behavior through automated natural language descriptions.
- MoE experts specialize in fine-grained tasks and functional transformations, rather than broad semantic domains or simple token processing, with this specialization becoming more pronounced in later layers.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.