Reproduce “Principled Coarse-Grained Acceptance for Speculative Decoding in Speech” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Moran Yanuka, Paul Dixon, Eyal Finkelshtein, Daniel Rotman, Raja Giryes
The Principled Coarse-Graining (PCG) framework enhances speculative decoding for autoregressive speech models by defining Acoustic Similarity Groups (ASGs) for perceptually interchangeable tokens, achieving a 1.4x speedup while significantly improving speech quality metrics compared to prior relaxation methods.
Key insights
- Relaxing exact token-level matching to group-level acceptance for acoustically similar tokens is essential for effective speculative decoding in speech generation.
- An overlap-aware coarse-graining strategy, combined with a coupled proposal and rejection sampling, can theoretically guarantee the fidelity of the output group distribution.
- The target model's internal embedding space proves more effective for defining perceptually relevant acoustic similarity groups than raw acoustic features like Mel-spectrograms.
- A practical implementation can retain the draft token upon group acceptance, prioritizing group-level acoustic fidelity over intra-group token-level exactness due to the inherent interchangeability of tokens within ASGs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.