Reproduce “UniPool: Learning Expert-to-Layer Ownership from Brief Global Access” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Minbin Huang, Han Shi, Chuanyang Zheng, Yimeng Wu, Guoxuan Chen, Xintong Yu, Yichun Yin, Hong Cheng

UniPool-Lock lets layers learn expert ownership through brief access to a shared pool, then locks each expert to one layer. Balanced pool-level routing and a target-cardinality regularizer encourage layer preferences. It beats vanilla MoE at all five tested scales; at 1.5B after 60B tokens, test loss falls from 1.6320 to 1.6073. This one-seed result does not isolate ownership as the sole cause.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.