Reproduce “Perceptual-Neural-Physical Sound Matching” on OpenResearch

About this paper

Han Han, Vincent Lostanlen, Mathieu Lagrange

Perceptual-Neural-Physical (PNP) loss is introduced as a learning objective for sound matching, designed to balance the computational efficiency of parameter-domain loss with the perceptual relevance of spectral loss. This method achieves comparable perceptual accuracy to P-loss and faster training than full spectral loss, particularly for synthetic percussion sounds, by employing a cached Riemannian metric to weight parameter deviations.

Key insights

• Spectral loss can be efficiently approximated by a perceptually weighted parameter loss, where the weights are determined by a pre-computable Riemannian metric, effectively bridging the gap between computational efficiency and perceptual accuracy.
• The empirical Riemannian metric M(θ) often exhibits ill-conditioning, necessitating an adaptive damping mechanism akin to the Levenberg-Marquardt algorithm to ensure stable convergence during neural network training.
• PNP loss intrinsically prioritizes the learning of perceptually relevant parameter dimensions, demonstrating improved retrieval accuracy for parameters whose changes induce larger perceptual differences, verified through analysis of the duration parameter.

Summary by alphaXiv.