Reproduce “U2++: Unified Two-pass Bidirectional End-to-end Model for Speech Recognition” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Di Wu, Binbin Zhang, Chao Yang, Zhendong Peng, Wenjing Xia, Xiaoyu Chen, Xin Lei
U2++ is an enhanced end-to-end speech recognition model that integrates a bidirectional attention decoder and a novel data augmentation technique (SpecSub) into the U2 framework, achieving a 5-10% relative Character Error Rate (CER) reduction over its predecessor. The model maintains unified streaming and non-streaming capabilities, demonstrating a 5.05% streaming CER on AISHELL-1 with 320ms latency, a competitive result for Chinese Mandarin speech recognition.
Key insights
- Integrating a bidirectional attention mechanism (L2R and R2L decoders) and combining their scores during a two-pass decoding strategy consistently leads to higher accuracy compared to unidirectional decoding.
- The proposed SpecSub data augmentation method independently contributes to notable improvements in model robustness and overall recognition accuracy.
- An equal distribution of layers between the L2R and R2L decoders (e.g., 3 layers each for a total of 6) yields optimal performance for the combined bidirectional architecture.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.