Reproduce “SingMOS-Pro: An Comprehensive Benchmark for Singing Quality Assessment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuxun Tang, Lan Liu, Wenhao Feng, Yiwen Zhao, Jionghao Han, Yifeng Yu, Jiatong Shi, Qin Jin
This work introduces SingMOS-Pro, a comprehensive multilingual and multi-task MOS dataset for automatic Singing Quality Assessment (SQA), comprising 7,981 clips with fine-grained annotations for overall, lyrics, and melody quality. The research establishes benchmarks demonstrating a significant domain gap between speech and singing quality assessment, showing that combining heterogeneous data with domain identifiers improves SQA model generalization.
Key insights
- A substantial domain gap exists between speech and singing quality assessment, as speech-trained MOS models demonstrate poor performance when applied to singing quality evaluation.
- Strategies like multi-dataset finetuning (MDF) and incorporating domain identifiers are effective in mitigating discrepancies from heterogeneous annotation standards and improving SQA model generalization.
- Simple integration of pitch information (MIDI pitch, pitch histograms) yielded only marginal performance improvements over plain self-supervised learning baselines for SQA, suggesting a need for more advanced melodic cue utilization.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.