Reproduce “GSLM2: Multi-Stream Speech Representation Learning and Generative Spoken Language Modeling” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yosuke Higuchi, Tom Fizycki, Mahi Luthra, Charles-Éric Saint-James, Azeez Syed, Angel Villar-Corrales, Mido Assran, Nicolas Ballas and 9 more
The authors introduce GSLM2, which models speech with unit streams from multiple encoder layers and predicts them together. In a matched 6,000-hour, 300-million-parameter comparison, it scores 76.3%, 67.2%, and 78.9% on lexical, syntactic, and semantic tests, above a single-stream model. The evidence is limited to tested benchmarks and estimated data scaling, not broad language ability.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.