Reproduce “Exploring Speaker Diarization with Mixture of Experts” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Gaobin Yang, Maokui He, Shutong Niu, Ruoyu Wang, Hang Chen, Jun Du
The research by Yang et al. develops a Neural Speaker Diarization with Memory-Aware Multi-Speaker Embedding and Sequence-to-Sequence (NSD-MS2S) architecture, further enhanced by a Shared and Soft Mixture of Experts (SS-MoE) module. This system achieved state-of-the-art results on competitive benchmarks like CHiME-7 and DIHARD-III, addressing issues of computational efficiency, speaker embedding robustness, and generalization across diverse acoustic conditions.
Key insights
- The Deep Interactive Module (DIM) within the Memory-Aware Multi-Speaker Embedding (MA-MSE) module significantly improves speaker embedding discriminability by performing multi-scale feature fusion.
- The Shared and Soft Mixture of Experts (SS-MoE) module, with its soft routing mechanism, effectively mitigates model bias and enhances generalization across diverse acoustic conditions.
- A parameter transfer strategy, initializing complex MoE models with pre-trained base model parameters, accelerates training convergence and reduces computational costs by over 50%.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.