Reproduce “NANSY++: Unified Voice Synthesis with Neural Analysis and Synthesis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hyeong-Seok Choi, Jinhyeok Yang, Juheon Lee, Hyeongju Kim
NANSY++, a collaboration between Seoul National University and Supertone Inc., introduces a unified analysis-synthesis framework for voice generation that disentangles voice into pitch, amplitude, linguistic content, and timbre. The system enables efficient and controllable voice synthesis across diverse applications, achieving high-quality reconstruction and competitive performance in tasks like text-to-speech and voice conversion with significantly reduced data requirements.
Key insights
- Voice can be effectively disentangled into fundamental attributes like pitch, amplitude, linguistic content, and timbre, enabling fine-grained control and modularity.
- Self-supervised learning techniques, including auto-encoding for F0 and contrastive loss with information perturbation for linguistic features, can robustly achieve disentanglement without requiring extensive annotated datasets.
- A unified analysis-synthesis architecture allows a single backbone network to serve diverse voice-related tasks, accelerating development and reducing training overhead for new applications.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.