Reproduce “From Tens of Hours to Tens of Thousands: Scaling Back-Translation for Speech Recognition” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Tianduo Wang, Lu Xu, Wei Lu, Shanbo Cheng

A study explores scaling speech back-translation, a technique utilizing text-to-speech models to create vast synthetic speech datasets from minimal real speech. This approach, leveraging tens of hours of real data to generate hundreds of thousands of synthetic hours, leads to a 30% average reduction in transcription error rates for multilingual automatic speech recognition systems.

Key insights

  • Minimal real speech data (tens of hours) can effectively bootstrap the generation of massive synthetic datasets (tens of thousands of hours) for ASR training.
  • The quality of the fine-tuned Text-to-Speech (TTS) model directly correlates with the performance improvement of Automatic Speech Recognition (ASR) models trained on its synthetic output.
  • The newly introduced Normalized Intelligibility (Norm_I) metric reliably predicts the utility of synthetic speech for ASR training by comparing ASR performance on real vs. synthetic data.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.