Reproduce “Kanade: A Simple Disentangled Tokenizer for Spoken Language Modeling” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zhijie Huang, Stephen McIntosh, Daisuke Saito, Nobuaki Minematsu

Kanade uses a compact stream of time-varying speech tokens and a separate utterance-level embedding for stable acoustic information such as speaker identity. Its self-supervised encoder supplies features to both paths; quantized content tokens and the embedding condition a decoder to reconstruct speech. On LibriSpeech test-clean, at 25 tokens/s, reconstruction WER is 2.4%, versus 2.7% for matched-rate k-means in this test.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.