Reproduce “Latent-MoE: Domain-Aware Mixture-of-Experts for PDEs with Multi-Regime Physics” on OpenResearch

About this paper

Hanwen Wang, Paris Perdikaris

Latent-MoE combines compact-support, domain-aware routing with shared neural backbones to improve physics-informed learning when PDE dynamics change across space or time. Across heterogeneous advection–diffusion and wave benchmarks, it reduced relative L2 errors to about 0.010–0.036, compared with 0.066–0.740 for the strongest competing architectures, while remaining competitive on homogeneous equations.

Key insights

• For ReLU coordinate networks, the local NTK magnitude can grow with coordinate magnitude, causing distant or late-stage regions to exert uneven influence during optimization; compact-support routing limits these interactions.
• Centering expert inputs is important: an uncentered global encoder produces greater gradient conflict, whereas shared centered encoders combine local conditioning with parameter reuse.
• Localization is most useful for multi-regime physics. On homogeneous Burgers, Allen–Cahn, and KdV controls, all tested models remain near the 10^-5 to 10^-4 error range, with no clear Latent-MoE advantage.

Summary by alphaXiv.