Reproduce “Detecting music deepfakes is easy but actually hard” on OpenResearch

About this paper

Darius Afchar, Gabriel Meseguer-Brocal, Romain Hennequin

Researchers at Deezer developed the first general-purpose music deepfake detector, demonstrating 99.8% accuracy under controlled conditions using amplitude spectrograms. While effective in ideal scenarios, the detector exhibited severe performance degradation when deepfake audios were subjected to common manipulations or generated by unseen architectures, highlighting the challenges of robust real-world deployment.

Key insights

• Detecting music deepfakes from known generators is surprisingly easy under controlled conditions, with a simple CNN achieving nearly perfect accuracy (99.8%).
• Detection performance drastically degrades when deepfake audios are subjected to common manipulations (e.g., pitch shift, noise, re-encoding) or come from generative models not seen during training, indicating a lack of real-world robustness and generalization.
• High accuracy scores alone are insufficient for reliable and ethical deployment; future deepfake detectors must prioritize robustness, generalization, calibration, and interpretability as essential features, rather than solely focusing on performance metrics.

Summary by alphaXiv.