Reproduce “Fast-VGAN: Lightweight Voice Conversion with Explicit Control of F0 and Duration Parameters” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Mathilde Abrassart, Nicolas Obin, Axel Roebel

Fast-VGAN, a lightweight, GAN-based voice conversion model, provides precise and explicit control over fundamental frequency, phoneme durations, intensity, and speaker identity. It achieves a 0.00% Word Error Rate for intelligibility and over 10 times real-time inference speed on a CPU, demonstrating robust performance even when synthesizing diverse expressive styles without specific training data.

Key insights

  • Explicit conditioning on interpretable features (F0, intensity, phoneme duration) enables fine-grained and direct control over expressive speech characteristics in voice conversion.
  • A lightweight, fully convolutional, GAN-based architecture can achieve real-time inference (over 10 times real-time on CPU for the generator) and high-quality speech generation, overcoming latency issues of other models.
  • The model can generalize to synthesize diverse expressive speech styles from neutral inputs, without being explicitly trained on expressive data, demonstrating robust disentanglement of prosody and speaker identity.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.