Reproduce “Codified audio language modeling learns useful representations for music information retrieval” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rodrigo Castellon, Chris Donahue, Percy Liang
Researchers at Stanford University found that representations from Codified Audio Language Modeling (CALM) systems, such as Jukebox, are highly effective for diverse discriminative Music Information Retrieval (MIR) tasks. This approach yielded a 30% relative performance improvement over traditional tag-based pre-training and significantly enhanced key detection capabilities.
Key insights
- Generative pre-training directly on codified music audio yields richer and more comprehensive representations for MIR tasks than pre-training based on human-provided labels or contrastive learning.
- Transformer language models, adapted from Natural Language Processing (NLP), are effective in learning general-purpose representations from structured audio sequences.
- Modeling the inherent structure of raw audio, rather than discrete labels, enables the capture of fundamental musical attributes like key information, which are often overlooked by label-centric pre-training.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.