Reproduce “Task-Aware Unified Source Separation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kohei Saijo, Janek Ebbers, François G. Germain, Gordon Wichern, Jonathan Le Roux
A unified neural network model is introduced that performs all major audio source separation tasks by employing learnable prompts to dynamically control separation behavior and the number of outputs. This approach resolves the issue of contradictory task goals in a single model, achieving performance comparable to or surpassing specialist models while offering enhanced flexibility in source extraction.
Key insights
- Learnable prompts can effectively condition a unified model to perform diverse and even contradictory source separation tasks (e.g., individual instruments vs. music-mix) within a single architecture.
- A Transformer-based cross-prompt module with positional encoding enables the model to handle a variable number of outputs and differentiate between multiple instances of the same source category (e.g., multiple speakers).
- Prompt dropout during training is crucial for maintaining robust performance when only a subset of sources is requested at inference time, enhancing practical flexibility.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.