Reproduce “Audio-Conditioned Diffusion LLMs for ASR and Deliberation Processing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mengqi Wang, Zhan Liu, Zengrui Jin, Guangzhi Sun, Chao Zhang, Philip C. Woodland
The Whisper-LLaDA framework integrates a Whisper acoustic encoder with a LLaDA diffusion LLM, demonstrating its utility in both refining ASR transcripts and serving as a standalone, faster decoder. It achieves a 12.3% relative Word Error Rate reduction on the LibriSpeech test-other set for deliberation-based processing and offers up to 2.4 times faster inference in standalone decoding compared to autoregressive baselines while maintaining competitive accuracy.
Key insights
- Acoustic features are critical for effective deliberation and ASR performance; a plain-text diffusion LLM without audio conditioning significantly degrades transcript quality.
- Diffusion LLMs, particularly with semi-autoregressive deliberation, can leverage bidirectional attention to effectively correct errors in initial ASR transcripts generated by unidirectional models.
- Diffusion-based and semi-autoregressive decoding strategies offer a favorable balance between efficiency and accuracy for standalone ASR, providing faster inference than autoregressive baselines with competitive recognition rates.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.