Reproduce “SpeechBlender: Speech Augmentation Framework for Mispronunciation Data Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yassine El Kheir, Shammur Absar Chowdhury, Ahmed Ali, Hamdy Mubarak, Shazia Afzal

Researchers from the Qatar Computing Research Institute introduced SpeechBlender, a speech augmentation framework that generates realistic mispronunciation errors by linearly interpolating raw speech signals of canonical and donor phonemes with fine-grained masks. The framework achieved a 5.0% gain in Pearson Correlation Coefficient (PCC) for English and a 4.63% increase in F1-score for Arabic over no-augmentation baselines, outperforming existing augmentation methods and state-of-the-art models in mispronunciation detection.

Key insights

  • Fine-grained linear interpolation of raw speech signals with diverse masks proves more effective for generating realistic L2 mispronunciation data than simpler 'Cut/Paste' augmentation methods.
  • The 'close phoneme-pair dictionary' and dynamic mixing factors (lambda values) allow for the generation of specific types of errors, from subtly accented phonemes to clearly mispronounced or missing units.
  • The framework's ability to create augmented mispronunciation data from correctly pronounced speech addresses the zero/few-shot problem in low-resource languages, expanding training sets without extensive manual annotation.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.