Reproduce “Enhancing Speaker Diarization with Large Language Models: A Contextual Beam Search Approach” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tae Jin Park, Kunal Dhawan, Nithin Koluguri, Jagadeesh Balam
NVIDIA researchers developed a contextual beam search approach that integrates large language models (LLMs) into speaker diarization systems, leveraging vast text data to significantly improve speaker attribution accuracy. This method achieves up to a 39.8% relative reduction in speaker-attributed word error rate (∆SA-WER) compared to acoustic-only baselines.
Key insights
- Large Language Models (LLMs) can effectively leverage vast text-only datasets to provide rich contextual and semantic cues for speaker attribution, which are typically inaccessible to acoustic-only systems.
- Integrating LLMs via a contextual beam search during decoding allows for a robust, modular system, similar to how LMs enhance ASR, overcoming data scarcity and fixed-speaker limitations of previous methods.
- Using a consistent type of language model for both speaker and word probability terms results in better performance, with LLMs showing superior ability to discern speaker identity from broader lexical context compared to n-grams.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.