Reproduce “CMT-LLM: Contextual Multi-Talker ASR Utilizing Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiajun He, Naoki Sawada, Koichi Miyazaki, Tomoki Toda
Researchers from Nagoya University and CyberAgent developed CMT-LLM, a unified framework that combines multi-talker automatic speech recognition with contextual biasing for rare word recognition by leveraging Large Language Models. The framework achieves improved Word Error Rates and significantly reduced Biased Word Error Rates on both simulated and real-world multi-speaker datasets by effectively integrating dynamic rare word lists into the LLM prompt via a novel filtering mechanism.
Key insights
- Large Language Models (LLMs) excel at global context modeling and long-range dependencies, making them suitable for multi-talker ASR and flexible integration of dynamic contextual information via prompts.
- Unifying multi-talker ASR and contextual biasing addresses a critical gap in real-world scenarios where both challenges are present and intertwined.
- Effective filtering of large contextual biasing lists is crucial for efficient and performant LLM-based contextual ASR, as excessively large prompts can degrade performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.