Reproduce “Faster IndexTTS-2: Accelerating and Streaming Autoregressive Zero-Shot Text-to-Speech Synthesis on GPUs” on OpenResearch

About this paper

Muyang Du, Shuang Yu, Junjie Lai

NVIDIA researchers developed Faster IndexTTS-2, an accelerated version of the IndexTTS-2 model, achieving up to 3.6x end-to-end speedup and 5.0x for the core GPT component compared to the PyTorch baseline, with minimal quality degradation. This work enables real-time, low-latency zero-shot text-to-speech synthesis on GPUs, integrating streaming and batched inference capabilities.

Key insights

• The autoregressive GPT component is the dominant bottleneck in IndexTTS-2, and its acceleration using TensorRT-LLM (5.0x speedup) is critical for achieving overall real-time performance.
• Careful adaptation of LLM-focused optimization frameworks (like TensorRT-LLM) is essential for their effective application to speech models, requiring specific modifications for conditioning, input handling, and intermediate state extraction.
• Streaming synthesis effectively reduces Time-To-First-Audio, with chunk size being a primary determinant of both latency and quality trade-offs, while batched inference significantly boosts throughput for offline scenarios.

Summary by alphaXiv.