Reproduce “Elastic Time: Dynamic Frame Rate Bottlenecks for Neural Audio Coding” on OpenResearch

About this paper

Dimitrios Bralios, Paris Smaragdis, Minje Kim

Elastic Time introduces a "Re-Bottleneck" framework to adapt pretrained fixed-frame-rate neural audio autoencoders into dynamic-frame-rate models, using a learned latent predictor to identify and omit redundant frames. This approach achieves comparable or superior reconstruction quality to fixed-rate baselines, outperforming other dynamic-rate methods like CodecSlime on music and speech, thereby mitigating computational and memory bottlenecks in long-context audio applications.

Key insights

• The learned latent predictor effectively identifies and exploits temporal redundancy within audio signals, allowing for shorter latent sequences without significant quality degradation.
• Both the greedy and dynamic programming chunking algorithms yield similar reconstruction quality, making the computationally efficient greedy approach practical for real-time applications.
• The "Re-Bottleneck" plug-in architecture allows efficient adaptation of pretrained autoencoders, avoiding costly retraining from scratch and enabling deployment-time rate control without external semantic supervision.

Summary by alphaXiv.