Reproduce “LoRP-TTS: Low-Rank Personalized Text-To-Speech” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Łukasz Bondaruk, Jakub Kubiak
Researchers from Samsung R&D Institute Poland developed LoRP-TTS, a method that uses Low-Rank Adaptation (LoRA) to enhance zero-shot personalized Text-to-Speech (TTS) systems. LoRP-TTS effectively adapts to new speakers using minimal, even noisy, real-world audio prompts, achieving improved speaker similarity and naturalness in synthesized speech while maintaining intelligibility.
Key insights
- LoRA provides a computationally efficient approach to speaker adaptation, introducing only 2.3% additional parameters, significantly reducing overhead compared to full fine-tuning.
- Effective speaker personalization can be achieved with a single, short (e.g., 3-second) audio sample and minimal adaptation steps (100 optimizer steps), outperforming methods using more noisy data from the same speaker.
- Simply increasing inference steps in the base TTS model does not effectively personalize the voice; targeted LoRA adaptation is crucial for improving speaker similarity, intelligibility, and overall quality in challenging scenarios.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.