Reproduce “Leveraging Neural Representations for Audio Manipulation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Scott H. Hawley, Christian J. Steinmetz
Researchers investigated how latent representations from pretrained neural network audio autoencoders encode common audio manipulations, finding that while these models capture some semantic structure for effects, the information is complex, nonlinearly organized, and not well disentangled across dimensions. Manipulation of effect parameters often results in nonlinear paths in the latent space, suggesting limitations for simple algebraic control.
Key insights
- Pretrained autoencoder latent spaces do encode some semantic structure related to audio effect classes, particularly when time-averaged representations are used.
- Varying a single audio effect parameter typically results in a complex, nonlinear trajectory through the latent space, limiting the effectiveness of simple linear operations for manipulation.
- The latent dimensions of the studied autoencoders exhibit a low level of disentanglement regarding audio manipulations, meaning effect information is distributed rather than isolated to specific dimensions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.