Reproduce “ALO-VC: Any-to-any Low-latency One-shot Voice Conversion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Bohan Wang, Damien Ronssin, Milos Cernak
ALO-VC introduces an any-to-any low-latency one-shot voice conversion system that achieves speech naturalness and speaker similarity competitive with non-causal state-of-the-art methods. The system operates with 55 ms latency and a Real-Time Factor (RTF) of 0.78 on a single CPU core, while being 3.2 times smaller in parameter count than VQMIVC.
Key insights
- Phonetic Posteriorgrams (PPGs) serve as an effective causal, speaker-independent linguistic content representation for constructing real-time voice conversion systems.
- Combining a statistical pitch predictor with causal positional encoding enables high speech quality and naturalness in a causal system with minimal future look-ahead.
- Leveraging robust, pre-trained speaker encoders significantly enhances speaker similarity in one-shot, any-to-any, low-latency voice conversion while preventing content leakage.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.