Reproduce “Multi-Loss Convolutional Network with Time-Frequency Attention for Speech Enhancement” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Liang Wan, Hongqing Liu, Yi Zhou, Jie Ji
A Multi-Loss convolutional Network with Time-Frequency Attention (MNTFA) is proposed, providing a computationally efficient model for speech enhancement. It utilizes axial self-attention for capturing long-range dependencies and a multi-objective loss function to achieve high perceptual quality and competitive ASR performance with significantly fewer parameters and GFLOPS compared to prior models.
Key insights
- Axial Self-Attention (ASA) efficiently models long-range dependencies across both time and frequency axes in spectrograms, offering a computationally lighter alternative to recurrent networks.
- A multi-objective loss function, combining time-domain MSE, multi-resolution STFT, and ASR (WavLM) losses, effectively balances perceptual speech quality with performance for downstream tasks like speech recognition.
- The integration of residual connections in the encoder-decoder structure aids in stable training and enables the model to focus on estimating noise components.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.