Reproduce “Layer-Scoped Expert-Budget Expansion Discovers Succinct Convergence in Sparse Mixture-of-Experts Reasoning: Reducing Reasoning Tokens at Matched Accuracy on MMLU-Pro” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Vincenzo Agrillo
The authors test a training-free routing change that admits additional experts in selected late layers of a sparse mixture-of-experts model. Tokens retain native top-8 routing elsewhere. On one model and benchmark, among 577 questions both configurations answered correctly, expansion reduced reasoning tokens by 8.5% and mean latency by 10.9%. Accuracy differed by three answers, but the comparison was statistically inconclusive.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.