Reproduce “Reproducible Evaluation of MoE Expert Caching: Replay Semantics, Workload Contamination, and Operating Regimes” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yu Zhang

A reproducibility study shows that MoE expert-cache results can change substantially with the replay contract, prompt construction, and workload operating regime. Event-atomic replay reverses some policy rankings, diversity-controlled prompts reduce apparent category locality, and most of the remaining Belady gap is tied to future victim selection rather than admission—where the tested causal predictor performs worse than LFRU.

Key insights

  • Sequential replay can prematurely evict experts needed later in the same fused event, selectively worsening LRU, LFRU, and Least-Stale and changing the apparent policy ranking.
  • Single-template workloads can manufacture locality: one category’s apparent 57.2% miss reduction fell to 5.9% after diversity controls, and only two of six category effects replicated clearly.
  • The union-to-capacity ratio is an important but incomplete descriptor: temporal ordering alone changed the oracle gap from 44.85% to roughly 30.8–31.1%, along with the best causal policy.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.