Reproduce “Achieving Human Parity in Conversational Speech Recognition” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
W. Xiong, J. Droppo, X. Huang, F. Seide, M. Seltzer, A. Stolcke, D. Yu, G. Zweig
Microsoft Research established an empirical human performance benchmark for conversational speech recognition on the NIST 2000 test set and demonstrated an automated system that achieves or slightly surpasses these human error rates. The system attained Word Error Rates of 5.8% on the Switchboard and 11.0% on the CallHome portions, utilizing advanced deep learning architectures and novel training methodologies.
Key insights
- Human performance in conversational speech recognition varies significantly across different subsets of conversational data (e.g., 5.9% WER on Switchboard vs. 11.3% on CallHome), invalidating the assumption of a single universal human error rate.
- The systematic application and combination of state-of-the-art deep learning models for both acoustic and language modeling, coupled with sophisticated training techniques, can enable ASR systems to achieve performance levels comparable to or exceeding human transcribers.
- Specific methodological advancements like spatial smoothing for LSTM acoustic models and lattice-free MMI sequence training provide substantial relative error reductions, confirming their effectiveness in improving ASR accuracy.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.