Reproduce “Non-Contrastive Self-Supervised Learning of Utterance-Level Speech Representations” on OpenResearch

About this paper

Jaejin Cho, Raghavendra Pappagari, Piotr Żelasko, Laureano Moro-Velazquez, Jesús Villalba, Najim Dehak

Researchers from Johns Hopkins University adapted the DINO non-contrastive self-supervised learning framework to create utterance-level speech representations from unlabeled data. This method achieved superior performance in unsupervised speaker verification compared to contrastive approaches and demonstrated the embeddings inherently capture emotion-related information.

Key insights

• Non-contrastive SSL (DINO) effectively overcomes the negative sampling challenge in speech, learning robust representations for identity-based tasks without explicit labels.
• DINO embeddings, trained in a completely unsupervised manner, capture both speaker identity and valuable emotion-predictive information, indicating their potential as general-purpose speech representations.
• A strong initial DINO model significantly improves the efficiency and performance of iterative pseudo-labeling pipelines for downstream tasks like speaker verification, requiring fewer iterations for convergence.

Summary by alphaXiv.