Reproduce “SimulMEGA: MoE Routers are Advanced Policy Makers for Simultaneous Speech Translation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chenyang Le, Bing Han, Jinshun Li, Songyong Chen, Yanmin Qian
SimulMEGA introduces a framework using Mixture-of-Experts (MoE) routers for unsupervised policy learning, converting offline autoregressive models into simultaneous generation systems for real-time speech translation and text-to-speech. It achieves a 0.3% to 3% BLEU degradation relative to its offline base, outperforming SeamlessM4T by consistently delivering better quality-latency trade-offs with zero additional inference-time computation.
Key insights
- Mixture-of-Experts (MoE) routers can effectively act as implicit, dynamic policy makers for simultaneous generation, leveraging both prefix-based and global contextual information.
- Policy learning for real-time streaming systems can be successfully achieved entirely during training, ensuring zero computational overhead during the critical inference phase.
- Precise control over gating score distributions, achieved through normalization and pre-Sigmoid noise, is essential for robust and consistent policy learning across diverse tasks and language pairs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.