Reproduce “Language-Statistical Analysis of Neural Audio Codec Tokens Across Architectures, Corpora, and Noise Conditions” on OpenResearch

About this paper

Joonyong Park, Shinnosuke Takamichi, David M. Chan, Shunsuke Kando, Yuki Saito, Hiroshi Saruwatari

Park et al. from The University of Tokyo and collaborators conducted a language-statistical analysis of Neural Audio Codec (NAC) tokens across 13 architectures, three corpora, and three noise conditions. The study found that NAC architecture and acoustic noise condition predominantly shaped token statistical properties, with codec families exhibiting distinct degradation signatures under noise. Jensen-Shannon Divergence (JSD) from clean to noisy unigram distributions negatively correlated with perceived speech quality (UTMOS) (r=-0.76) and positively with F0 error (r=0.57) under real-world noise, offering a decoder-free diagnostic.

Key insights

• NAC architecture and acoustic condition are the dominant factors influencing token statistical properties (e.g., Zipf ",alpha", Heaps ",beta", entropy), while corpus identity has negligible influence.
• Different NAC architectures exhibit distinct degradation patterns under noise; multi-codebook RVQ codecs tend to "collapse" under white noise and "explode" under DEMAND noise, while single-codebook VQ codecs remain "neutral".
• Jensen-Shannon Divergence (JSD) between clean and noisy token distributions serves as an effective decoder-free diagnostic for codec quality degradation, correlating with perceived naturalness and F0 errors under real-world noise.

Summary by alphaXiv.