Reproduce “RemixIT: Continual self-training of speech enhancement models via bootstrapped remixing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Efthymios Tzinis, Yossi Adi, Vamsi Krishna Ithapu, Buye Xu, Paris Smaragdis, Anurag Kumar
RemixIT introduces a self-supervised method for training speech enhancement models using only in-domain noisy mixtures, eliminating the need for isolated clean speech or noise waveforms. By combining bootstrapped remixing with continual teacher-student refinement, the approach achieves state-of-the-art performance in low-resource settings and demonstrates robust domain adaptation.
Key insights
- Bootstrapped remixing enables robust learning from imperfect pseudo-targets by decorrelating the student's errors from the teacher's errors, allowing the student model to effectively refine its performance beyond the teacher's initial capabilities.
- Continuous and dynamic teacher refinement, particularly through sequential updates, is crucial for sustained performance improvement in self-training, as it gradually enhances the quality of generated pseudo-targets.
- High-performance speech enhancement models can be adapted to diverse real-world acoustic environments (unsupervised, semi-supervised, zero-shot) by leveraging only in-domain noisy mixtures, significantly reducing data collection burdens.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.