Reproduce “BAST: Binaural Audio Spectrogram Transformer for Binaural Sound Localization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sheng Kuang, Jie Shi, Kiki van der Heijden, Siamak Mehrkanoon
The Binaural Audio Spectrogram Transformer (BAST) provides a hierarchical, pure attention-based deep learning model for binaural sound localization. It achieves an angular distance error of 1.29 degrees, outperforming prior CNN-based and other Transformer models, by effectively capturing global acoustic features and drawing inspiration from human auditory processing.
Key insights
- Pure Transformer architectures, when structured as a dual-stream, neurobiologically-inspired model, can effectively capture global acoustic features and achieve superior binaural sound localization.
- The choice of binaural integration method (e.g., subtraction) and loss function (hybrid) significantly impacts localization accuracy, especially for models with non-shared stream parameters.
- Attention analysis reveals that the model learns to prioritize specific low-frequency and early-time cues for localization, offering interpretability into the underlying feature processing.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.