Reproduce “MBTFNet: Multi-Band Temporal-Frequency Neural Network For Singing Voice Enhancement” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Weiming Xu, Zhouxuan Chen, Zhili Tan, Shubo Lv, Runduo Han, Wenjiang Zhou, Weifeng Zhao, Lei Xie
MBTFNet is a two-stage neural network for singing voice enhancement that removes noise, background music, and backing vocals. It introduces an Implicit Personalized Enhancement (IPE) stage that dynamically adapts to the target singer's voice without explicit enrollment, yielding superior performance over existing speech enhancement and music source separation models.
Key insights
- Singing voice enhancement necessitates a dedicated approach that accounts for the distinct acoustic characteristics of singing and complex interference, including backing vocals, beyond what general SE or MSS models provide.
- Combining multi-band decomposition with specialized inter-band and intra-band temporal-frequency modeling improves the differentiation of target singing voices from harmonically correlated background music.
- An implicit personalization mechanism that dynamically extracts and updates speaker embeddings based on a predicted signal cleanliness score is highly effective and practical, outperforming explicit enrollment for enhancing target voices from mixed audio.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.