Reproduce “Uni-ASR: Unified LLM-Based Architecture for Non-Streaming and Streaming Automatic Speech Recognition” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yinfeng Xia, Jian Tang, Junfeng Hou, Gaopeng Xu, Haitao Yao
Alibaba researchers developed Uni-ASR, a unified LLM-based architecture that supports both non-streaming and streaming automatic speech recognition within a single model. This framework achieves competitive non-streaming accuracy while providing robust streaming performance, with a novel fallback decoding strategy reducing average Word Error Rate by 11-20% compared to greedy decoding in streaming modes.
Key insights
- A unified LLM-based architecture can intrinsically support both high-accuracy non-streaming and low-latency streaming ASR.
- Modeling cross-chunk contextual dependencies through a context-aware training paradigm and a specific decoding strategy is essential for robust streaming performance.
- The "latest-token fallback" decoding strategy significantly improves streaming recognition accuracy, especially with shorter speech chunks, while maintaining minimal computational overhead.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.