Reproduce “AudioSet-R: A Refined AudioSet with Multi-Stage LLM Label Reannotation” on OpenResearch

About this paper

Yulin Sun, Qisheng Xu, Yi Su, Qian Zhu, Yong Dou, Xinwang Liu, Kele Xu

Researchers at the National University of Defense Technology developed AudioSet-R, a refined version of the AudioSet benchmark, by applying a multi-stage, LLM-driven reannotation framework to enhance label quality and granularity. This improved dataset consistently led to performance gains across various audio classification models, with the AST model's mAP increasing by 3.2 percentage points and SSAST's mAP increasing by 4.6 percentage points over original baselines.

Key insights

• Chaining multiple large language and multimodal models allows for complex, automated data curation, moving beyond single-pass or human-dependent annotation.
• Enhancing label accuracy and semantic granularity in datasets consistently improves the performance of diverse audio classification models, including both supervised and self-supervised architectures.
• The quality of both training and evaluation datasets significantly impacts model assessment; improved validation labels reveal truer model capabilities, while noisy original labels can obscure performance.

Summary by alphaXiv.