Reproduce “Environmental Sound Deepfake Detection Challenge: An Overview” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Han Yin, Yang Xiao, Rohan Kumar Das, Jisheng Bai, Ting Dang
The inaugural Environmental Sound Deepfake Detection (ESDD) Challenge established a new benchmark for detecting manipulated environmental audio, driven by a novel large-scale dataset. Top-performing systems achieved Equal Error Rates (EERs) as low as 0.25%, a considerable improvement over baseline systems' EERs of 12-15%.
Key insights
- Self-supervised learning (SSL) models, utilized as front-ends, are highly effective in extracting robust acoustic representations that generalize to unseen deepfake generators.
- Sophisticated back-end classifiers, coupled with targeted data augmentation techniques and ensemble methods, significantly enhance deepfake detection performance and robustness.
- Detection of deepfakes from undisclosed video-to-audio generators with minimal training data (Track 2) surprisingly yielded lower EERs than generalization to unseen text-to-audio/audio-to-audio generators (Track 1), indicating strong generalization by top models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.