Reproduce “A Survey of Foundation Models for Music Understanding” on OpenResearch

About this paper

Wenjun Li, Ying Cai, Ziyang Wu, Wenyi Zhang, Yifan Chen, Rundong Qi, Mengqi Dong, Peigen Chen and 8 more

This survey provides a comprehensive review of recent foundation models and multimodal large language models applied to music understanding, empirically evaluating their capabilities across music classification and captioning tasks. It identifies the strengths and limitations of current models and proposes future research directions to enhance musical intelligence.

Key insights

• General multimodal language models (e.g., Qwen-Audio, LTU) often outperform music-specific models due to comprehensive pre-training on diverse, large-scale multimodal datasets and robust architectures.
• Model performance is significantly influenced by training data quality, multi-task supervised fine-tuning, and the potential for modality interference when integrating too many input types.
• Current foundation models lack deep, rule-based musical knowledge, often prioritizing generative tasks over understanding, which limits their ability to provide professionally reasoned outputs.

Summary by alphaXiv.