Reproduce “CosyAccent: Duration-Controllable Accent Normalization Using Source-Synthesis Training Data” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Qibing Bai, Shuhao Shi, Shuai Wang, Yukai Ju, Yannan Wang, Haizhou Li
Researchers from The Chinese University of Hong Kong, Nanjing University, and Tencent Ethereal Audio Lab introduced CosyAccent, a model that utilizes a novel "source-synthesis" training data method to perform accent normalization. This approach yields superior content preservation, reducing Word Error Rate to 12.96% compared to baselines, and enhances naturalness and speaker similarity, all while being trained exclusively on synthetic accented speech.
Key insights
- Inverting the synthetic data generation paradigm by synthesizing the L2 source (instead of the L1 target) from authentic L1 data effectively circumvents L1-L2 data scarcity and avoids TTS-induced artifacts in training targets.
- A robust content encoder with CTC loss, combined with a DiT decoder and explicit speaker embeddings, is essential for preserving linguistic content, naturalness, and speaker identity in accent normalization.
- Achieving flexible and duration-controllable accent normalization in NAR models is possible by leveraging position scaling within Rotary Positional Encoding to align source and target lengths.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.