Reproduce “Diff-MST: Differentiable Mixing Style Transfer” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Soumya Sai Vanka, Christian Steinmetz, Jean-Baptiste Rolland, Joshua Reiss, George Fazekas
The DIFF-MST framework automates multi-track music mixing by transferring production style from a reference song using a differentiable mixing console and a transformer controller. It predicts explicit audio effect parameters, achieving lower average AF Loss (0.0326) and Fréchet Audio Distance (15.1) on large-scale evaluations compared to existing baselines, indicating improved style replication and audio quality.
Key insights
- Grounding deep learning models in differentiable traditional audio effects provides interpretability and controllability, crucial for professional audio production workflows.
- Transformer architectures offer an effective solution for handling variable numbers of input tracks in complex audio processing tasks like multi-track mixing.
- Developing a custom Audio Feature (AF) loss, incorporating perceptually relevant Music Information Retrieval (MIR) features, is more effective for guiding mixing style transfer than generic audio similarity metrics alone.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.