Reproduce “ProGRes: Prompted Generative Rescoring on ASR n-Best” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ada Defne Tur, Adel Moumen, Mirco Ravanelli
ProGRes introduces a method for improving ASR accuracy by dynamically augmenting n-best hypothesis lists with LLM-generated candidates and then interpolating acoustic and linguistic scores for rescoring. This approach yielded relative Word Error Rate (WER) reductions up to 24.7% on standard datasets, demonstrating the benefit of combining generative LLMs with ASR for enhanced transcription.
Key insights
- Interpolating LLM linguistic scores with ASR acoustic scores is crucial for performance, as neither an LLM-only rescoring nor a prompt-only approach yielded comparable improvements.
- Instruction-tuned LLMs can effectively generate linguistically plausible corrections, especially when initial ASR hypotheses are subtly inaccurate, leading to greater relative gains.
- The PROGRES method achieves its performance in a zero-shot manner, demonstrating the inherent generalization capabilities of pre-trained LLMs without specific fine-tuning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.