Reproduce “Automatic Pronunciation Assessment using Self-Supervised Speech Representation Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Eesung Kim, Jae-Jin Jeon, Hyeji Seo, Hoon Kim
Researchers at Kakao Enterprise's AI Lab demonstrated the superior performance of self-supervised learning (SSL) models, specifically fine-tuned HuBERT Large, for automatic pronunciation assessment. Their method achieved a state-of-the-art Pearson Correlation Coefficient of 0.82 for holistic pronunciation scores on the KESL dataset, significantly outperforming traditional and other deep learning baselines.
Key insights
- Self-supervised learning models, particularly when fine-tuned on non-native speech, inherently learn rich contextual representations highly beneficial for pronunciation assessment.
- Higher transformer layers within SSL models encode more abstract, pronunciation-relevant information, and combining representations from all layers yields superior results.
- A BLSTM-based regression module effectively leverages both the extracted SSL audio features and linguistic text embeddings for accurate pronunciation scoring.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.