Reproduce “Whisper: Courtside Edition Enhancing ASR Performance Through LLM-Driven Context Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yonathan Ron, Shiri Gilboa, Tammuz Dubnov
Reichman University researchers developed a multi-agent LLM pipeline called "Whisper: Courtside Edition" to enhance automatic speech recognition performance for domain-specific content without retraining the underlying ASR model. The system achieved a statistically significant 17.0% relative reduction in Word Error Rate on NBA basketball commentary by generating contextual prompts that guide Whisper's decoder.
Key insights
- Intelligent prompt-based augmentation using a multi-agent LLM pipeline can significantly improve domain-specific ASR performance by guiding the decoder with external contextual knowledge without requiring ASR model retraining.
- Robust decision-filtering logic, implemented by dedicated LLM agents, is crucial to prevent over-correction and maintain transcription stability, ensuring that proposed corrections improve rather than degrade output quality.
- Decomposing the context generation task into specialized agents for topic identification, named entity correction, and jargon extraction allows for targeted intervention and effective utilization of the ASR model's prompt mechanism.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.