Reproduce “FastPitchFormant: Source-filter based Decomposed Modeling for Speech Synthesis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Taejun Bak, Jae-Sung Bae, Hanbin Bae, Young-Ik Kim, Hoon-Young Cho
FastPitchFormant (FPF) introduces a decomposed neural text-to-speech model, designed by NCSOFT's Speech AI Lab, which applies source-filter theory to mel-spectrogram generation. This architecture independently models formant and excitation components, allowing for robust pitch manipulation with improved audio quality and speaker characteristic preservation compared to prior methods.
Key insights
- Speech production can be effectively decomposed into independent source (excitation) and filter (vocal tract/formant) components.
- By explicitly separating the generation of formant-related (linguistic) and excitation-related (prosodic) representations in a neural TTS model, pitch can be manipulated without corrupting phonetic content or speaker timbre.
- Principles from classical speech science, like source-filter theory, can inform and improve the architectural design of modern deep learning models for speech synthesis.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.