Reproduce “Preference Optimization for Non-Verbal Vocalization Synthesis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haoyang Li, Chenglin Xu, Junchuan Zhao, Yuang Cao, Liumeng Xue, Yiwen Guo, Eng Siong Chng
Researchers at Nanyang Technological University and LIGHTSPEED systematically investigated applying preference optimization to enhance non-verbal vocalization (NV) synthesis within large language model-based Text-to-Speech systems. Their approach, utilizing a new NV-aware Character Error Rate (NV-CER) metric for preference learning, consistently improved the accuracy of both non-verbal and lexical content in synthesized speech across multiple evaluation modalities.
Key insights
- A dedicated NV-aware preference signal, such as the proposed NV-CER, is essential for effectively guiding preference optimization to improve non-verbal vocalization synthesis in TTS.
- Constructing preference pairs using the best-scoring and worst-scoring synthetic candidates based on NV-CER (Syn+/Syn-) proved most effective, outperforming methods involving ground-truth or NV-free synthetic speech.
- The NV-CER's w_NV weighting mechanism provides a controllable means to prioritize improvements in NV accuracy over lexical accuracy, allowing for application-specific trade-offs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.