Reproduce “On the information rate of phones, syllables and words” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Herman Kamper
The authors compare how limited language models encode the same English speech sequences using phones, syllables, words, and multiword units. They train n-gram and transformer models on LibriSpeech and measure held-out coding cost in bits per second. In a 20M-parameter-body transformer, word units reach 28.07 bps versus 29.56 for phones; this result is limited to the data and model setups studied.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.