Reproduce “Wave-Trainer-Fit: Neural Vocoder with Trainable Prior and Fixed-Point Iteration towards High-Quality Speech Generation from SSL features” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hien Ohnaka, Yuma Shirahata, Masaya Kawamura
WaveTrainerFit is a neural vocoder developed by researchers from NAIST and LY Corporation that enhances speech generation from self-supervised learning (SSL) features by incorporating trainable priors and a reference-aware gain adjustment mechanism. It consistently outperforms baselines in objective metrics and speaker similarity, demonstrating improved quality from fewer inference steps and robustness across various SSL feature layers.
Key insights
- Learning an explicit prior distribution for initial noise sampling from SSL features significantly reduces the complexity of the denoising task and leads to higher quality speech generation from early iterations.
- Directly guiding the trainable prior to learn target speech energy enables a reference-aware gain adjustment mechanism, offloading this task from the main waveform generation network and improving overall consistency.
- Applying trainable priors in the time-frequency domain is crucial for reducing sequence length and modeling complexity while effectively utilizing the spectral characteristics of speech.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.