Reproduce “Investigating End-to-End ASR Architectures for Long Form Audio Transcription” on OpenResearch

About this paper

Nithin Rao Koluguri, Samuel Kriman, Georgy Zelenfroind, Somshubra Majumdar, Dima Rekesh, Vahid Noroozi, Jagadeesh Balam, Boris Ginsburg

NVIDIA researchers evaluated end-to-end Automatic Speech Recognition architectures for single-pass offline transcription of long-form audio, finding that a fine-tuned Fast Conformer with Limited Context Attention and a Global Token achieved the best Word Error Rate across various datasets while processing hours of audio. The study also quantified efficiency and accuracy trade-offs between CTC and RNNT decoders, revealing CTC's robustness but RNNT's higher accuracy with specific model configurations.

Key insights

• Integrating global context, either through Squeeze-and-Excitation modules or a dedicated Global Token, is crucial for improving transcription accuracy on long-form audio.
• The quadratic complexity of full attention severely limits the maximum processable audio length (e.g., 12 minutes for original Conformer), highlighting the need for efficient attention mechanisms like Limited Context Attention.
• While RNNT decoders can achieve higher absolute accuracy with specific, sophisticated architectures (e.g., fine-tuned Fast Conformer with GT), CTC decoders offer superior efficiency (approximately 43x faster) and greater robustness across various long-audio durations.

Summary by alphaXiv.