Reproduce “Serialized Output Training by Learned Dominance” on OpenResearch

About this paper

Ying Shi, Lantian Li, Shi Yin, Dong Wang, Jiqing Han

Researchers from Harbin Institute of Technology, Tsinghua University, Beijing University of Posts and Telecommunications, and Huawei Technologies Co., Ltd. introduce Dominance-Based Serialized Output Training (DOM-SOT) to improve multi-talker Automatic Speech Recognition (ASR). This method autonomously learns to order speaker transcriptions based on intrinsic speech properties, outperforming existing serialization strategies like Permutation Invariant Training (PIT) and First-In-First-Out (FIFO) across various overlap conditions.

Key insights

• The model can autonomously learn to identify and prioritize 'dominant' speech components based on factors like loudness and gender, offering a multi-factor bias that is more robust than single, predefined biases.
• Learning a dynamic, intrinsic ordering logic through a dominance score (CTC loss) provides a more generalizable and stable serialization strategy compared to static rules or instantaneous loss minimization.
• The auxiliary serialization module can be discarded during inference, allowing for efficient deployment of the trained ASR model without added computational overhead.

Summary by alphaXiv.