Reproduce “Streaming End-to-End Multilingual Speech Recognition with Joint Language Identification” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chao Zhang, Bo Li, Tara Sainath, Trevor Strohman, Sepand Mavandadi, Shuo-yiin Chang, Parisa Haghani
Google LLC researchers developed a streaming end-to-end multilingual Automatic Speech Recognition (ASR) system that jointly performs language identification (LID), achieving a 2nd-pass Word Error Rate (WER) of 7.8% across nine language locales, comparable to oracle LID systems. This model delivers 92.9% average streaming LID accuracy while adding less than 0.5% to total model parameters.
Key insights
- Jointly training ASR with a lightweight, frame-synchronous LID predictor can achieve ASR accuracy on par with systems provided with oracle language identification.
- The 0.9-second right-context inherent in the cascaded encoder's second-pass is effectively utilized to perform accurate streaming LID without introducing additional latency for the primary ASR output.
- Directly feeding predicted one-hot LID vectors to the second-pass decoder, allowing gradient backpropagation, is more effective for ASR WER reduction than integrating them into the right-context encoder or stopping gradients.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.