Reproduce “Wave-U-Mamba: An End-To-End Framework For High-Quality And Efficient Speech Super Resolution” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yongjoon Lee, Chanwoo Kim
Wave-U-Mamba presents an end-to-end framework for speech super-resolution directly on raw waveforms, integrating Mamba blocks into a U-Net GAN architecture. The model achieves state-of-the-art perceptual quality, scoring a Mean Opinion Score (MOS) of 4.01, while demonstrating over nine times faster inference and using less than 2% of the parameters compared to previous leading methods.
Key insights
- Direct waveform-to-waveform processing using a U-Net with Mamba blocks effectively preserves phase information and captures long-term dependencies, overcoming limitations of mel-spectrogram-based approaches.
- A relatively low-dimensional bottleneck in the generator helps constrain the propagation of extraneous patterns from FFT-interpolated low-resolution signals, improving reconstruction quality.
- Including an L1 waveform loss in training can degrade performance and introduce metallic artifacts, suggesting that spectral and adversarial losses are more effective for perceptual quality in speech generation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.