Reproduce “Reverb Conversion of Mixed Vocal Tracks Using an End-to-end Convolutional Deep Neural Network” on OpenResearch

About this paper

Junghyun Koo, Seungryeol Paik, Kyogu Lee

Researchers at Seoul National University developed an end-to-end convolutional deep neural network for converting musical reverb between mixed vocal tracks, capable of transferring reverb characteristics or performing de-reverberation. The system achieved a 64.8% subjective preference for ground truth spatial similarity and improved objective metrics like PESQ (3.105) and STOI (80.77%) in reverb conversion tasks.

Key insights

• An end-to-end deep learning approach can effectively convert musical reverb between mixed vocal tracks, a task not previously addressed by deep neural networks in this specific context.
• The integration of adversarial training (GAN) significantly contributes to generating more perceptually realistic reverberant sounds, leading to improved objective performance in reverb conversion.
• The model demonstrates stronger performance in converting 'wetter' input tracks to 'drier' targets, suggesting potential limitations in generating complex wetter reverb from magnitude-only spectrograms.

Summary by alphaXiv.