Reproduce “Kimi-Audio Technical Report” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

KimiTeam, Ding Ding, Zeqian Ju, Yichong Leng, Songxiang Liu, Tong Liu, Zeyu Shang, Kai Shen and 32 more

Kimi-Audio is an open-source audio model that processes speech and other sounds and can respond with text or speech. It combines discrete semantic tokens with continuous acoustic features, then uses language-model layers and separate text and audio output heads. On LibriSpeech, its word error rates are 1.28 and 2.42, below Qwen2.5-Omni’s 2.37 and 4.21; it does not lead every benchmark.

Key insights

  • Low frequency tokenization (12.5Hz) effectively bridges the gap between audio and text sequence lengths
  • Continual pretraining from existing LLMs combined with carefully designed audio-text tasks improves performance
  • Chunk-wise streaming with look-ahead mechanism enables real-time applications

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.