Reproduce “ASR Error Detection via Audio-Transcript entailment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Nimshi Venkat Meripo, Sandeep Konam
Researchers at Abridge AI Inc. introduced an end-to-end method for Automatic Speech Recognition (ASR) error detection, framing the problem as an audio-transcript entailment task. This approach, which jointly encodes raw audio and hypothesized text, reduced the classification error rate for general ASR errors by 12% and for medical errors by 15.4% compared to a confidence-score based baseline.
Key insights
- Framing ASR error detection as an audio-transcript entailment task offers an effective, ASR-system-independent approach by assessing bidirectional consistency between the two modalities.
- Fusing raw acoustic features from self-supervised models with linguistic representations significantly improves error detection performance, demonstrating the value of multimodal information for identifying discrepancies.
- The proposed method shows enhanced performance in detecting errors within domain-specific medical terms, particularly for frequently occurring terms and those in the "Chemicals & Drugs" semantic group, suggesting distinct patterns that aid recognition.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.