Reproduce “Real-time Timbre Remapping with Differentiable DSP” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jordie Shier, Charalampos Saitis, Andrew Robertson, Andrew McPherson
This research introduces a data-driven framework for real-time timbre remapping, which explicitly translates relative timbral changes from an acoustic input to a synthesizer's parameters using a novel feature difference loss within a Differentiable Digital Signal Processing (DDSP) setup. The method effectively remapped timbral variations from acoustic snare drums to a synthetic drum, enabling nuanced and reactive sonic outputs in a live performance context.
Key insights
- Optimizing for relative differences in audio features, rather than absolute values, effectively captures the perceptual concept of 'timbre analogies' and is crucial for preserving musical expressivity in timbre remapping.
- Differentiable Digital Signal Processing (DDSP) provides a robust framework to directly optimize complex synthesizer parameters based on audio feature differences, bypassing the need for explicit parameter supervision.
- Real-time timbre remapping can be achieved with relatively lightweight neural network models, which allows for efficient training and practical implementation in low-latency musical performance systems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.