Reproduce “Automatic Quality Estimation for ASR System Combination” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shahab Jalalvand, Matteo Negri, Daniele Falavigna, Marco Matassoni, Marco Turchi
Reference-free quality estimation is used to order ASR hypotheses dynamically before applying ROVER, allowing black-box recognizer outputs and distant-microphone transcriptions to be combined at the speech-segment level. The method lowers WER from 26.1% for random-order ROVER to 18.6% on CHiME-3 and from 14.6–12.4% to a best of 11.9% on IWSLT, while also learning how many hypotheses to include.
Key insights
- Local quality ranking is more useful than a single recording-wide order because different recognizers and microphones perform differently across acoustic conditions.
- Combining fewer, better-ranked hypotheses generally outperforms voting over all candidates; low-quality outputs can introduce errors despite increasing system diversity.
- Quality estimation can improve transcription fusion without decoder internals, but its effectiveness depends on training–test similarity and the diversity of candidate hypotheses.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.