Reproduce “Transformer-based Online CTC/attention End-to-End Speech Recognition Architecture” on OpenResearch

About this paper

Haoran Miao, Gaofeng Cheng, Changfeng Gao, Pengyuan Zhang, Yonghong Yan

Researchers from the Institute of Acoustics, Chinese Academy of Sciences, developed a Transformer-based online end-to-end speech recognition architecture that integrates streaming encoder and decoder designs. The architecture achieved a 23.66% Character Error Rate (CER) with 320ms latency on the HKUST Mandarin benchmark, demonstrating a 4.18% absolute improvement over prior online LSTM-based models.

Key insights

• The State Reuse Chunk-SAE significantly improves computational efficiency (1.5x faster) and extends the effective receptive field by reusing hidden states, leading to better contextual understanding for streaming encoders.
• Future context (right-side context) in the streaming encoder contributes more to online ASR performance than historical context (left-side context), though increasing historical context can effectively reduce required future context and thus latency.
• The MTA-based SAD successfully enables streaming output generation for Transformer decoders by intelligently truncating encoder outputs while maintaining sufficient context for accurate decoding.

Summary by alphaXiv.