Reproduce “TF-MossFormer: Integrating Convolution Gated Local-Global Attentions for Enhanced Time-Frequency Domain Monaural Speech Separation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shengkui Zhao, Zexu Pan, Haoxu Wang, Biao Tian, Bin Ma, Xiangang Li
TF-MossFormer, developed by researchers at Token Foundry, Alibaba Group, introduces a time-frequency domain transformer for monaural speech separation that integrates content-aware sliding-window local attention, global attention, and convolutional gating. This architecture achieves state-of-the-art performance on the WSJ0-2Mix dataset, reaching 24.4 dB SI-SDRi with the largest model configuration.
Key insights
- Optimal speech separation performance is achieved when local attention is applied before global attention (V1 configuration), suggesting a beneficial hierarchy for processing fine-grained details prior to broader contextual integration.
- Content-aware sliding-window attention with specific, moderate window lengths (e.g., 31 temporal frames and 7 frequency bands) is crucial for balancing local detail capture and computational efficiency, outperforming full global attention or rigid static convolutions.
- The inclusion of convolutional gating mechanisms significantly contributes to performance gains by improving feature selection and facilitating robust information flow within the hybrid local-global attention architecture.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.