Reproduce “SC-SOT: Conditioning the Decoder on Diarized Speaker Information for End-to-End Overlapped Speech Recognition” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuta Hirano, Sakriani Sakti
Researchers at Nara Institute of Science and Technology (NAIST) developed SC-SOT, a framework that improves end-to-end overlapped speech recognition by explicitly conditioning the decoder of a Serialized Output Training (SOT) model with speaker diarization information. The system achieves relative Word Error Rate reductions of up to 9.5% on Libri2Mix and 10.4% on Libri3Mix, particularly when leveraging oracle speaker activity information.
Key insights
- The decoder in conventional SOT models plays a crucial role in implicitly separating speakers within overlapped speech, as evidenced by its non-monotonic source-target attention patterns.
- Explicitly providing speaker identity (embeddings) and activity information to the ASR decoder significantly enhances its ability to disentangle mixed speech and improve transcription accuracy.
- The full potential of combining both speaker embeddings and activity information for conditioning is constrained by the accuracy of the *predicted* diarization module, suggesting a need for improved joint training strategies.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.