Reproduce “Note-Level Singing Melody Transcription for Time-Aligned Musical Score Generation” on OpenResearch

About this paper

Leekyung Kim, Sungwook Jeon, Wan Heo, Jonghun Park

An end-to-end deep learning framework, T3MS, is introduced for directly transcribing singing melodies into time-aligned musical scores by recognizing note onset, offset, pitch, and note value from polyphonic audio. The framework demonstrates competitive F1 scores for note-level transcription and superior performance in time-aligned note value recognition compared to quantization-based methods.

Key insights

• Joint estimation of note onset, offset, pitch, and note value within a unified framework leads to improved overall transcription accuracy through mutual reinforcement of these inter-related attributes.
• Directly predicting discrete note values as part of the end-to-end model surpasses post-transcription quantization methods in achieving higher accuracy and more musically coherent outputs.
• Pseudo-labeling techniques can effectively address data scarcity for complex music transcription tasks by leveraging existing, less comprehensively annotated datasets to generate approximate but useful labels for training.

Summary by alphaXiv.