Reproduce “InferGrad: Improving Diffusion Models for Vocoder by Considering Inference in Training” on OpenResearch

About this paper

Zehua Chen, Xu Tan, Ke Wang, Shifeng Pan, Danilo Mandic, Lei He, Sheng Zhao

Researchers at Imperial College London and Microsoft developed INFERGRAD, a training methodology for diffusion models that integrates fast inference directly into the training objective. This approach enables diffusion models to generate high-quality audio with significantly fewer steps, achieving a 3x speedup with equivalent perceptual quality compared to prior methods.

Key insights

• Directly incorporating an "Infer Loss" during training, which evaluates the quality of samples generated using few inference steps, effectively bridges the gap between model training and its practical deployment.
• Utilizing perceptually-aligned metrics like multi-resolution STFT loss, including both magnitude and phase information, is crucial for achieving high-fidelity and artifact-free audio synthesis during few-step inference.
• Training the model with a range of inference schedules, rather than a single fixed one, significantly enhances the model's robustness and reduces its sensitivity to specific inference parameter choices.

Summary by alphaXiv.