Reproduce “Voice conversion with limited data and limitless data augmentations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Olga Slizovskaia, Jordi Janer, Pritish Chandna, Oscar Mayor
This research from Voctro Labs and Voicemod investigates how data augmentation techniques can reduce the need for extensive target speaker data in real-time voice conversion systems. It demonstrates that sophisticated audio-effect augmentations improve conversion quality and target similarity, particularly showing benefits for male target voices, while operating within strict latency constraints.
Key insights
- Increasing raw target speaker data does not consistently improve conversion quality or similarity, indicating that data efficiency and augmentation strategies may be more impactful than data volume.
- Sophisticated audio-effect augmentations, such as spectral envelope transformations, are more effective than simple noise additions, which often degrade audio quality.
- Male target voices consistently achieve higher quality and similarity scores than female targets, pointing to inherent acoustic or modeling challenges specific to female voice conversion.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.