Reproduce “Accent Vector: Controllable Accent Manipulation for Multilingual TTS Without Accented Data” on OpenResearch

About this paper

Thanathai Lertpetchpun, Thanapat Trachu, Jihwan Lee, Tiantian Feng, Dani Byrd, Shrikanth Narayanan

Researchers at the University of Southern California developed "Accent Vector," a framework to enable controllable accent manipulation in multilingual Text-to-Speech (TTS) systems without requiring accented training data. This approach leverages fine-tuning on native-language speech to induce L2 accents, achieving fine-grained control over accent strength and allowing for the linear composition of multiple accents while preserving speaker identity.

Key insights

• Accent characteristics can be effectively encoded as a linear parameter shift (Accent Vector) within a pretrained TTS model, enabling manipulation through simple vector arithmetic.
• Leveraging high-resource native-language speech data for fine-tuning a multilingual TTS model allows for inducing L2 accents without the need for large, accent-specific L2 speech corpora.
• The concept of task vectors, extended to speech accents, provides a unified mechanism for fine-grained, continuous control over both segmental and suprasegmental features of an accent, and supports the composition of multiple accents.

Summary by alphaXiv.