Reproduce “VarArray: Array-Geometry-Agnostic Continuous Speech Separation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Takuya Yoshioka, Xiaofei Wang, Dongmei Wang, Min Tang, Zirun Zhu, Zhuo Chen, Naoyuki Kanda
Microsoft researchers developed VarArray, an array-geometry-agnostic continuous speech separation neural network that unifies inter-channel phase difference features, Conformer blocks, and Transform-Average-Concatenate layers to leverage nonlinear channel correlations. The model achieved lower Word Error Rates, such as 17.7% on AMI-dev (8-channel), outperforming both channel-independent and geometry-specific baselines in real meeting transcription tasks.
Key insights
- Integrating TAC layers and IPD features with Conformer blocks allows the model to effectively leverage nonlinear correlations between input channels in an array-geometry-agnostic manner.
- Training on a diverse dataset encompassing multiple microphone array geometries significantly enhances the model's generalization capabilities and overall performance.
- End-to-end optimization using ASR-based loss, even with a small amount of real meeting data, effectively mitigates the simulation-to-real domain mismatch and yields consistent WER improvements across both overlap and non-overlap speech segments.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.