Reproduce “Towards Word-Level End-to-End Neural Speaker Diarization with Auxiliary Network” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yiling Huang, Weiran Wang, Guanlong Zhao, Hank Liao, Wei Xia, Quan Wang
Researchers at Google LLC developed a Word-level End-to-End Neural Diarization (WEEND) system that integrates automatic speech recognition (ASR) and speaker diarization within a single RNN-T architecture, using an auxiliary network. This joint model improved word diarization error rate (WDER) by 25% on 5-minute Callhome data and by up to 56% on 30-second Fisher segments compared to a modular baseline, directly producing speaker-attributed transcripts.
Key insights
- Freezing the parameters of a pre-trained ASR model while training an auxiliary diarization network efficiently adapts the system without degrading core ASR performance.
- Extracting speaker features from intermediate layers (e.g., 5th Conformer layer) of the ASR encoder is more effective than using raw features or the final encoder output.
- Sharing blank logits between the ASR and auxiliary diarization tasks within the RNN-T framework enables inherent synchronization of word recognition and speaker labeling.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.