Reproduce “Kimi-Audio Technical Report” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
KimiTeam, Ding Ding, Zeqian Ju, Yichong Leng, Songxiang Liu, Tong Liu, Zeyu Shang, Kai Shen and 32 more
Kimi-Audio is an open-source audio model that processes speech and other sounds and can respond with text or speech. It combines discrete semantic tokens with continuous acoustic features, then uses language-model layers and separate text and audio output heads. On LibriSpeech, its word error rates are 1.28 and 2.42, below Qwen2.5-Omni’s 2.37 and 4.21; it does not lead every benchmark.
Key insights
- Low frequency tokenization (12.5Hz) effectively bridges the gap between audio and text sequence lengths
- Continual pretraining from existing LLMs combined with carefully designed audio-text tasks improves performance
- Chunk-wise streaming with look-ahead mechanism enables real-time applications
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.