Reproduce “ZipCodec: Ultra-Low-Frame-Rate Streaming Speech Coding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Luca Della Libera, Cem Subakan, Mirco Ravanelli
ZipCodec is a causal neural speech codec that compresses audio into discrete representations at 6.25 Hz—eight times fewer frames than 50 Hz streaming codecs—while maintaining a 0.80 kbps bitrate. Across reconstruction and downstream evaluations, it preserves speech content, speaker identity, and paralinguistic information, while running faster than real time on an Intel i7-10875H CPU with 124.68 ms p99 streaming latency.
Key insights
- Frame rate and bitrate are separate design dimensions: ZipCodec matches the bitrate of comparable systems while reducing the number of transmitted representations by up to eight times relative to 50 Hz streaming codecs.
- Distilling WavLM representations enables the compact causal code to retain information useful for reconstruction, speaker recognition, emotion recognition, keyword spotting, and intent classification—not only waveform detail.
- The factorized scalar quantizer achieves complete code usage without requiring an explicit enormous codebook, supporting efficient low-rate discrete representations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.