Reproduce “CabinSep: IR-Augmented Mask-Based MVDR for Real-Time In-Car Speech Separation with Distributed Heterogeneous Arrays” on OpenResearch

About this paper

Runduo Han, Yanxin Hu, Yihui Fu, Zihan Zhang, Yukai Jv, Li Chen, Lei Xie

CabinSep, a lightweight neural mask-based MVDR approach for real-time in-car speech separation, significantly reduces ASR error rates in multi-speaker scenarios. It achieved a 17.5% relative reduction in average Character Error Rate (CER) compared to state-of-the-art baselines while maintaining high computational efficiency at 0.4 GMACs and an RTF of 0.21 on an in-car CPU.

Key insights

• Applying MVDR beamforming exclusively during inference effectively maintains the distortionless property of separated speech, crucial for ASR compatibility, while avoiding numerical instabilities during neural network training.
• Computational efficiency for real-time in-car deployment can be achieved by strategically reducing processing for computationally intensive modules (e.g., "time skip" in TAC) and integrating advanced architectures (e.g., Conformer in sub-bands) in an optimized manner.
• A specific data augmentation strategy combining simulated and real-recorded Impulse Responses, particularly by using real IRs for the speaker's target zone, is highly effective for improving speaker localization accuracy in acoustically complex in-car environments.

Summary by alphaXiv.