Reproduce “Synthetic Data Domain Adaptation for ASR via LLM-based Text and Phonetic Respelling Augmentation” on OpenResearch

About this paper

Natsuo Yamashita, Koichi Nagatsuka, Hiroaki Kokubo, Kota Dohi, Tuan Vu Ho

Researchers at Hitachi, Ltd. developed an ASR domain adaptation framework that utilizes large language models to generate diverse synthetic training data and introduce phonetic variability at the text level. This framework reduced the overall Word Error Rate (WER) by up to 2.5% and the Biased WER (B-WER) for domain-specific terms by over 40% on various domain-specific datasets, outperforming existing synthetic data-based domain adaptation methods.

Key insights

• Selecting synthetic training data based on a balance of lexical diversity, domain-specific complexity (higher perplexity), and targeted term coverage significantly improves ASR performance for specialized domains.
• Introducing realistic phonetic variability by generating orthographic respellings at the text level is more effective for ASR domain adaptation than post-synthesis acoustic augmentations like SpecAugment, especially for human-recorded audio.
• Over-generating text with LLMs and then applying a multi-objective filtering strategy is crucial for producing high-quality synthetic data that effectively covers domain-specific vocabulary and fosters robustness.

Summary by alphaXiv.