Reproduce “Leveraging Soft Distributions of SSL-Derived Discrete Speech Tokens for Downstream Inference” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kentaro Onda, Satoru Fukayama, Daisuke Saito, Nobuaki Minematsu
The method trains downstream speech models on hard HuBERT tokens, then uses probability-weighted token embeddings at inference without retraining the speech encoder. This retains hard-token training while addressing information lost through nearest-centroid selection. On ERJ, HuBERT with 1,024 clusters achieves 49.4% WER, versus 51.1% for hard inference and 50.5% for continuous features. This exception is limited to three ERJ settings; continuous features generally perform best.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.