Reproduce “Phoneme-Guided Initialization for LLM-based Speech Recognition” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ryo Magoshi, Shinsuke Sakai, Tatsuya Kawahara
The authors introduce phoneme-to-grapheme (P2G) initialization for end-to-end speech recognition, preparing an audio encoder to map speech to phonemes and an LLM to map phonemes to writing before connecting and fine-tuning them on paired speech and transcripts. On 10-hour AISHELL-1, the model lowers character error rate from 39.1% to 11.1%, but gains are limited on Tatar and Urdu.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.