Reproduce “Label-Looping: Highly Efficient Decoding for Transducers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Vladimir Bataev, Hainan Xu, Daniel Galvez, Vitaly Lavrukhin, Boris Ginsburg
The label-looping algorithm redefines greedy decoding for Transducer-based Automatic Speech Recognition models, optimizing batched inference by redesigning prediction loops. This approach achieves up to 2.1X total inference speedup and 3.9X non-encoder computation speedup, notably enabling the first efficient and exact batched decoding for Token-and-Duration Transducers.
Key insights
- Decoupling time advancement across batch elements by focusing on non-blank predictions maximizes parallelism and reduces idle GPU cycles.
- Calling the prediction network only for non-blank emissions, instead of every frame, significantly reduces redundant computations and wasted predictor state updates.
- Leveraging CUDA tensor-based data structures for batched hypotheses allows for highly efficient, parallel execution of operations on GPUs, translating algorithmic redesign into practical speedups.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.