Reproduce “Separate-and-Detect: Unified Drum Transcription and Stem Generation via Latent Diffusion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wei-Han Hsu, Chih-Cheng Chang, Bo-Yu Chen, Li Su, Yi-Hsuan Yang
This research from National Taiwan University and Academia Sinica developed a unified "separate-and-detect" pipeline for drum transcription and stem generation using a multi-track latent diffusion model. The system provides both symbolic drum events and editable acoustic stems for five drum components, significantly improving transcription accuracy for kick and snare drums while outperforming existing baselines in source separation quality.
Key insights
- A generative demixing approach using a "separate-and-detect" pipeline can serve as an effective and interpretable front end for symbolic drum transcription, offering superior performance for foundational drums compared to direct end-to-end ADT.
- The introduction of training-only auxiliary branches, particularly the Onset Branch, can significantly improve downstream transcription utility by regularizing the latent space to prioritize features essential for onset detection, without negatively impacting reconstruction quality.
- There exists a decoupling between signal reconstruction quality (mel-MSE), generative distributional quality (FAD), and discrete transcription utility, indicating that models can be optimized for specific analytical tasks beyond raw signal fidelity.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.