Reproduce “AudioTurbo: Fast Text-to-Audio Generation with Rectified Diffusion” on OpenResearch

About this paper

Junqi Zhao, Jinzheng Zhao, Haohe Liu, Yun Chen, Lu Han, Xubo Liu, Mark Plumbley, Wenwu Wang

AudioTurbo implements rectified diffusion for text-to-audio generation, enabling high-quality audio synthesis with as few as 3 sampling steps by leveraging pre-trained diffusion models to generate training data for direct noise-to-data mapping. This method dramatically accelerates inference speed while preserving audio quality and text-audio alignment.

Key insights

• Pre-trained diffusion models can be leveraged to generate paired noise-data samples, which serve as training data for learning efficient, direct transport paths using rectified diffusion.
• Rectified diffusion transforms the traditional multi-step stochastic diffusion process into a deterministic ordinary differential equation, allowing for much faster sampling with fewer inference steps.
• By training only the core UNet of the latent diffusion model while freezing other pre-trained components, AudioTurbo preserves the quality and capabilities of existing high-performing TTA models.

Summary by alphaXiv.