Reproduce “Accelerating Transducers through Adjacent Token Merging” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuang Li, Yu Wu, Jinyu Li, Shujie Liu
Adjacent Token Merging (A-ToMe) is introduced to enhance the computational efficiency of Transformer-based end-to-end Automatic Speech Recognition (ASR) transducers. This method adaptively reduces acoustic token sequence length by merging adjacent tokens, achieving up to a 70% inference speedup on GPU with minimal impact on Word Error Rate.
Key insights
- Acoustic tokens become increasingly redundant in deeper Transformer layers, indicated by higher cosine similarity between adjacent tokens, making them amenable to merging.
- Preserving the temporal order of acoustic tokens through *adjacent* merging is crucial for ASR, a key distinction from general token merging methods used in other domains.
- Adaptive and gradual subsampling, achieved by stacking A-ToMe modules, allows for variable output frame rates that can better accommodate the diverse durations of acoustic units in speech.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.