Reproduce “Crossing the Species Divide: Transfer Learning from Speech to Animal Sounds” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jules Cauzinille, Marius Miron, Olivier Pietquin, Masato Hagiwara, Ricard Marxer, Arnaud Rey, Benoit Favre
This research systematically evaluates the transferability of self-supervised speech models to diverse bioacoustic detection and classification tasks, demonstrating that representations learned from human speech can effectively generalize to animal vocalizations. The study finds that models pre-trained with noise robustness achieve competitive performance with specialized bioacoustic models and often outperform traditional machine learning approaches.
Key insights
- Pre-trained speech models capture generalized acoustic properties transferable across a wide range of animal taxa, with performance not significantly affected by phylogenetic proximity to humans.
- Models incorporating noise robustness during pre-training, such as WavLM and XEUS, generally exhibit superior performance in bioacoustic transfer tasks due to the noisy nature of real-world bioacoustic recordings.
- Lower to mid-level transformer layers (e.g., layers 3-11 for HuBERT, 4-15 for WavLM) are optimal for bioacoustic transfer, as deeper layers tend to encode more speech-specific, abstract features.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.