Reproduce “REBORN: Reinforcement-Learned Boundary Segmentation with Iterative Training for Unsupervised ASR” on OpenResearch

About this paper

Liang-Hsuan Tseng, En-Pei Hu, Cheng-Han Chiang, Yuan Tseng, Hung-yi Lee, Lin-shan Lee, Shao-Hua Sun

REBORN, an unsupervised ASR framework from National Taiwan University, introduces a reinforcement-learned segmentation model that iteratively optimizes segment boundaries alongside a phoneme prediction model. This approach achieves state-of-the-art Phoneme Error Rates on LibriSpeech (8.3% PER) and TIMIT, outperforming previous methods and showing strong performance across multiple languages.

Key insights

• Iteratively learning segmentation directly optimized for phoneme prediction leads to more acoustically meaningful boundaries and superior UASR performance compared to static or separately derived methods.
• The reinforcement-learned segmentation model tends to identify finer-grained acoustic units, potentially smaller than conventional phonemes, which are then effectively processed and de-duplicated by the phoneme prediction model.
• A reward function combining perplexity difference with regularization terms (edit distance, length difference) is crucial for stably guiding the non-differentiable segmentation learning process towards improved ASR outputs.

Summary by alphaXiv.