Reproduce “Real Time Speech Enhancement in the Waveform Domain” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Alexandre Defossez, Gabriel Synnaeve, Yossi Adi
Facebook AI Research developed a causal, real-time speech enhancement model based on the DEMUCS architecture that operates directly on raw audio waveforms. This model achieves state-of-the-art performance on benchmarks, runs efficiently on a laptop CPU with a Real-Time Factor (RTF) of 0.6, and improves ASR Word Error Rate by up to 51% in noisy conditions.
Key insights
- High-quality speech enhancement can be effectively achieved purely in the raw waveform domain, bypassing complexities of explicit spectral transformation and phase estimation.
- The DEMUCS architecture, originally for music source separation, demonstrates versatility and strong performance when adapted for speech enhancement.
- Combining time-domain L1 loss with a multi-resolution STFT loss is crucial for optimizing both objective signal fidelity and subjective perceptual quality.
- Specific waveform-domain data augmentations, particularly `Revecho` for simulating reverberation, are essential for improving model robustness to diverse real-world acoustic conditions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.