Reproduce “Kanade: A Simple Disentangled Tokenizer for Spoken Language Modeling” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhijie Huang, Stephen McIntosh, Daisuke Saito, Nobuaki Minematsu
Kanade uses a compact stream of time-varying speech tokens and a separate utterance-level embedding for stable acoustic information such as speaker identity. Its self-supervised encoder supplies features to both paths; quantized content tokens and the embedding condition a decoder to reconstruct speech. On LibriSpeech test-clean, at 25 tokens/s, reconstruction WER is 2.4%, versus 2.7% for matched-rate k-means in this test.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.