Reproduce “DDSynth-RL: Audio Synthesizer Inversion via Discrete Diffusion with Reinforcement Learning” on OpenResearch

About this paper

Tristan Wu, Daniel Chin, Junan Zhang, Junyan Jiang, Yansen Jing, Gus Xia

The DDSYNTH-RL framework addresses synthesizer inversion by combining discrete diffusion for generative parameter modeling with reinforcement learning for black-box optimization. This approach yields substantial improvements in out-of-domain audio matching, notably reducing wMFCC distance from 10.04 to 5.96 on the NSynth dataset.

Key insights

• Discrete diffusion effectively models complex, discrete synthesizer parameter spaces, avoiding the order dependency of autoregressive models and the continuous relaxation mismatch of flow matching.
• Reinforcement learning, specifically GRPO, provides an effective mechanism for optimizing generative models for perceptual audio quality using black-box audio feedback.
• The two-stage approach leverages supervised pretraining for robust initial parameter prediction and RL fine-tuning for improved direct auditory matching, particularly for out-of-domain data.

Summary by alphaXiv.