Reproduce “Provably Learning Mixtures of Linear Experts” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Adrian Candocia, Aamod Varma, Mark Davenport
The authors analyze gradient EM for Gaussian-cluster mixtures with linear experts. The method trains gates and experts jointly on likelihood. In simulations with six true experts and sixteen fitted experts, effective routed-expert counts are 6.09 for uniform mixtures and 3.82 for skewed mixtures. The theorem guarantees convergence in polynomially many iterations for population training under its separation and initialization assumptions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.