Reproduce “SoundMorpher: Perceptually-Uniform Sound Morphing with Diffusion Model” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xinlei Niu, Jing Zhang, Charles Patrick Martin
The method SoundMorpher enables perceptually uniform sound morphing by integrating a pre-trained latent diffusion model with a novel Sound Perceptual Distance Proportion (SPDP). It generates seamless and natural audio transitions across diverse sound types, including musical instruments, environmental sounds, and music, demonstrating improved performance over prior approaches.
Key insights
- Linear interpolation of morphing factors does not inherently result in perceptually uniform transitions; an explicit perceptual distance metric (SPDP) is crucial for natural sound morphing.
- Log Mel-spectrograms effectively capture the pseudo-3D perceptual and semantic information needed for robust sound similarity assessment in morphing contexts.
- Model adaptation, specifically using LoRA with an optimal rank (e.g., r=4), significantly enhances morphing correspondence and quality by fine-tuning the pre-trained diffusion model to source/target specifics.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.