Reproduce “Leveraging Self-Supervised Learning for Speaker Diarization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiangyu Han, Federico Landini, Johan Rohdin, Anna Silnova, Mireia Diez, Lukas Burget
Brno University of Technology's Speech@FIT demonstrates that leveraging the self-supervised WavLM model substantially reduces the labeled data required for neural speaker diarization training, achieving new state-of-the-art performance on far-field datasets like AMI and AISHELL-4. The approach, which integrates WavLM with a Conformer architecture, shows that large amounts of simulated data are often unnecessary and can even be detrimental.
Key insights
- WavLM-based systems are significantly more robust to data scarcity, maintaining high performance even with substantially less training data.
- Simulated data can negatively impact the performance of WavLM-based diarization systems, especially when real data is available, challenging conventional training practices.
- Fine-tuning WavLM parameters jointly with the Conformer-based EEND module yields superior results, indicating the benefits of task-specific adaptation for self-supervised representations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.