Reproduce “Trainingless Adaptation of Pretrained Models for Environmental Sound Classification” on OpenResearch

About this paper

Noriyuki Tonami, Wataru Kohno, Keisuke Imoto, Yoshiyuki Yajima, Sakiko Mishima, Reishi Kondo, Tomoyuki Hino

Researchers from NEC Corporation and Doshisha University developed a trainingless method for adapting pre-trained environmental sound classification models, improving their robustness to challenging unseen domains like Distributed Fiber-Optic Sensor (DFOS) data by directly manipulating time-frequency features in intermediate layers without requiring GPU resources or backpropagation.

Key insights

• Audio Spectrogram Transformers (ASTs) implicitly preserve time-frequency (TF-ish) structures within their intermediate layers, which can be explicitly recovered and manipulated.
• Direct manipulation of these internal TF-ish representations, using classical signal processing principles like frequency filtering, enables effective domain adaptation without requiring gradient-based training or substantial computational resources.
• The effectiveness of this direct manipulation is dependent on the specific AST architecture, with some models (e.g., PaSST) maintaining TF-ish fidelity deeper into the network than others (e.g., MS-CLAP).

Summary by alphaXiv.