Reproduce “Robust Neural Audio Fingerprinting using Music Foundation Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Shubhr Singh, Kiran Bhat, Xavier Riley, Benjamin Resnick, John Thickstun, Walter De Brouwer

Leveraging pretrained music foundation models (MuQ, MERT) as backbones for neural audio fingerprinting, this study by SoundPatrol and Cornell University achieved improved robustness against diverse audio manipulations, with the MuQ-Unfrozen model showing an 88.18% track-level hit rate and 96% accuracy for challenging Encodec compression.

Key insights

  • Pretrained music foundation models (MuQ, MERT) serve as significantly more effective and robust backbones for neural audio fingerprinting compared to models trained from scratch or general-purpose audio models.
  • Fine-tuning the encoder weights of music foundation models (unfrozen) generally leads to improved performance and adaptability in audio fingerprinting tasks.
  • Comprehensive data augmentation strategies, especially those mimicking aggressive real-world distortions like Encodec compression and temporal shifts, are critical for developing highly resilient fingerprinting systems.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.