Reproduce “Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Suyoun Kim, Takaaki Hori, Shinji Watanabe
A joint CTC-attention based end-to-end speech recognition model is presented, employing multi-task learning to enhance the robustness and training efficiency of attention-based systems. This approach achieves consistently lower Character Error Rates, for instance, 7.36% on WSJ1 and 44.99% on the noisy CHiME-4 dataset, outperforming standalone attention models by 5.4-14.6% without external language models.
Key insights
- Combining CTC and attention via multi-task learning leverages CTC's monotonic alignment strength to stabilize attention model training, improving robustness in noisy conditions.
- The CTC objective acts as an effective regularization and guide, accelerating the learning of proper alignments for the attention mechanism without needing heuristic windowing.
- A relatively low weighting for the CTC loss (optimal = 0.2) proved most effective, allowing the attention model's sequence-to-sequence capabilities to dominate overall accuracy while benefiting from CTC's guidance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.