Reproduce “BridgeVoC: Revitalizing Neural Vocoder from a Restoration Perspective” on OpenResearch

About this paper

Andong Li, Tong Lei, Rilin Chen, Kai Li, Meng Yu, Xiaodong Li, Dong Yu, Chengshi Zheng

BridgeVoC, developed by researchers from CAS, Tencent AI Lab, and Tsinghua University, redefines neural vocoding as an audio restoration task within a Data-to-Data diffusion framework, significantly reducing inference steps while achieving state-of-the-art audio quality. The model employs a novel subband-aware network and enables high-fidelity single-step generation, outperforming prior diffusion models that require many more steps for comparable results.

Key insights

• Vocoding from a Mel-spectrum exhibits similar rank reduction to audio restoration tasks (like low-pass filtering), suggesting that a Data-to-Data diffusion paradigm starting from a degraded spectral surrogate is more efficient than generating from pure noise.
• Shifting from a Noise-to-Data to a Data-to-Data diffusion framework, specifically using the Schr{o}dinger Bridge, significantly shortens the generation trajectory and improves the upper bound of performance, enabling high quality with very few inference steps.
• Explicitly leveraging hierarchical subband priors through an uneven frequency division and efficient time-frequency contextual modeling with large-kernel convolutions (BCD network) leads to superior spectral reconstruction and computational efficiency.

Summary by alphaXiv.