Trainingless Adaptation of Pretrained Models for Environmental Sound Classification

TL;DR

The paper tackles robustness of pretrained environmental sound classifiers to unseen domains while minimizing compute by proposing a trainingless adaptation method. It first reveals TF-ish structures in intermediate layers of Audio Spectrogram Transformer (AST) models and then introduces a non-gradient frequency filtering approach that operates on unfolded TF-like representations (Fold/Unfold) to suppress non-informative frequency bands. Validation on ESC-50 with a DFOS-like domain shift shows substantial gains, notably up to $20.40$ percentage points in accuracy for PaSST at $-15$ dB SNR, demonstrating that simple signal-processing steps can robustify DNNs without retraining. This approach enables rapid, resource-efficient domain adaptation and demonstrates a practical path to integrate legacy signal processing with modern DNN architectures.

Abstract

Deep neural network (DNN)-based models for environmental sound classification are not robust against a domain to which training data do not belong, that is, out-of-distribution or unseen data. To utilize pretrained models for the unseen domain, adaptation methods, such as finetuning and transfer learning, are used with rich computing resources, e.g., the graphical processing unit (GPU). However, it is becoming more difficult to keep up with research trends for those who have poor computing resources because state-of-the-art models are becoming computationally resource-intensive. In this paper, we propose a trainingless adaptation method for pretrained models for environmental sound classification. To introduce the trainingless adaptation method, we first propose an operation of recovering time--frequency-ish (TF-ish) structures in intermediate layers of DNN models. We then propose the trainingless frequency filtering method for domain adaptation, which is not a gradient-based optimization widely used. The experiments conducted using the ESC-50 dataset show that the proposed adaptation method improves the classification accuracy by 20.40 percentage points compared with the conventional method.

Figures (7)

• Figure 1: Concept of TF-ish structures in intermediate layer of DNN
• Figure 2: Proposed frequency filtering method on TF-ish domain of intermediate layers
• Figure 3: Signals used for confirmation of TF-ish features in intermediate layers
• Figure 4: Time and frequency continuity in intermediate layers of AST models
• Figure 5: Randomness in intermediate layers of AST models