Reproduce “Learning to Solve Inverse Problems for Perceptual Sound Matching” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Han Han, Vincent Lostanlen, Mathieu Lagrange
Researchers at Nantes Université developed the Perceptual–Neural–Physical (PNP) loss, a novel objective function for training deep neural networks to perform Perceptual Sound Matching (PSM) efficiently. This method achieves state-of-the-art parameter retrieval for nonstationary and inharmonic sounds by leveraging the Joint Time-Frequency Scattering Transform (JTFS) and a precomputed Riemannian metric, leading to a 100-fold speedup in training.
Key insights
- Precomputing the Riemannian metric allows the use of computationally expensive, perceptually accurate features like JTFS within DNN training by providing a 100-fold speedup during gradient descent.
- JTFS is indispensable for achieving high-performance perceptual sound matching with nonstationary and inharmonic sounds, significantly outperforming conventional spectral representations like MSS.
- Reparameterization of synthesizer outputs, alongside regularizations such as weight decay and gradient clipping, plays a crucial role in stabilizing training and improving convergence to optimal perceptual matches.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.