Reproduce “Voice Conversion for Whispered Speech Synthesis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Marius Cotescu, Thomas Drugman, Goeric Huybrechts, Jaime Lorenzo-Trueba, Alexis Moinet
This study from Amazon.com developed and evaluated methods for converting normally phonated speech into whispered speech while preserving speaker identity. Deep Neural Network (DNN) voice conversion models achieved high naturalness and intelligibility, demonstrating strong generalization to unseen speakers and enabling the newly released Whisper Mode in Amazon Alexa.
Key insights
- Deep Neural Networks (DNNs) can effectively learn the complex spectral transformations required to convert normal speech to whispered speech while largely preserving speaker identity, even without pitch cues.
- Diverse multi-speaker training datasets are crucial for achieving robust generalization of DNN voice conversion models to unseen speakers and varied acoustic conditions.
- Cross-gender whispered speech conversion remains challenging, indicating that gender-specific acoustic characteristics in the cepstral domain necessitate gender-balanced training data for optimal performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.