Reproduce “Multiscale Audio Spectrogram Transformer for Efficient Audio Classification” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wentao Zhu, Mohamed Omar
Amazon researchers introduced the Multiscale Audio Spectrogram Transformer (MAST), which processes audio spectrograms hierarchically to achieve both higher classification accuracy and improved efficiency. The model achieved a 22.2% top-1 accuracy increase on Kinetics-Sounds over AST while using 5 times fewer MACs and 42% fewer parameters.
Key insights
- Audio events inherently possess a hierarchical structure, and explicitly modeling this multiscale nature within Transformer architectures can significantly improve performance and representation learning.
- Integrating pooling operations into Transformer blocks effectively reduces computational overhead and parameters while enabling the extraction of more discriminative, hierarchical features.
- Successful hierarchical Transformer designs from computer vision (e.g., Swin, MViT) can be effectively adapted to the audio domain, leading to both efficiency and accuracy gains.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.