Reproduce “Hierarchical Sequence to Sequence Voice Conversion with Limited Data” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Praveen Narayanan, Punarjay Chakravarty, Francois Charette, Gint Puskorius
Researchers from Ford Greenfield Labs developed a hierarchical sequence-to-sequence approach for voice conversion that addresses the challenge of limited parallel training data. Their method combines autoencoder pretraining on single-speaker data with transfer learning on a small parallel dataset, achieving plausible voice conversion results.
Key insights
- Pretraining on abundant single-speaker data allows the model to learn robust speech representations and temporal alignment before cross-speaker conversion.
- A hierarchical recurrent encoder effectively distills linguistic content and removes speaker-specific temporal redundancies from input spectrograms.
- Transfer learning with a reduced learning rate enables effective adaptation of pretrained models to the cross-speaker voice conversion task with limited parallel data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.