Reproduce “UniMoMo: Expert Merging-Based MoE Acceleration for Large Recommendation Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lei Xin, Bin Gu, Peize Li, Zitong Wang, Jianbo Zhao, Changjiang Jiang, Chao Huang, Xuyang Zhao and 3 more
UniMoMo converts a trained sparse mixture-of-experts recommender to a smaller standard model using unlabeled calibration interactions. It profiles expert behavior on shared states, uses routing exposure to protect high-traffic experts, merges admissible groups, and then applies supervised adaptation. Across nine dataset-depth settings, two- and four-expert models retained near-source NDCG@10 while reducing measured latency, under RankMixer and single-GPU conditions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.