Reproduce “ARCHI-TTS: A flow-matching-based Text-to-Speech Model with Self-supervised Semantic Aligner and Accelerated Inference” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chunyat Wu, Jiajun Deng, Zhengxi Liu, Zheqi Dai, Haolin He, Qiuqiang Kong
ARCHI-TTS introduces a non-autoregressive text-to-speech model utilizing flow matching, incorporating a self-supervised semantic aligner for robust text-speech consistency and a training-free strategy for accelerated inference. The model achieves a 1.98% Word Error Rate and 0.70 Speaker Similarity on LibriSpeech-PC, outperforming several state-of-the-art systems while delivering a Real-Time Factor of 0.09 at 32 NFE steps.
Key insights
- A self-supervised semantic aligner can robustly establish text-speech consistency, adapting to diverse content without rigid duration modeling, which is vital for natural speech.
- Training-free reuse of condition encoder features across denoising steps offers a practical and efficient method to accelerate inference in flow-matching models without requiring complex distillation.
- Employing highly compressed, low-token-rate speech latent representations, coupled with global speaker embeddings, allows for a simplified and unified synthesis pipeline.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.