Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Performance Comparison of CNN and AST Models with Stacked Features for Environmental Sound Classification

Parinaz Binandeh Dehaghania, Danilo Penab, A. Pedro Aguiar

TL;DR

The paper tackles environmental sound classification (ESC) under data- and resource-constrained conditions by evaluating CNNs trained on stacked feature representations (Log-Mel, MFCC, GTCC, Chroma, Spectral Contrast, Tonnetz) and comparing them to an Audio Spectrogram Transformer (AST) across ESC-50 and UrbanSound8K with varying pretraining. It demonstrates that feature-stacked CNNs offer data- and compute-efficient performance, with CNN-1 achieving up to 92.46% validation accuracy on ESC-50 when pre-trained on ESC-50 and fine-tuned on UrbanSound8K, while AST benefits substantially from large-scale pretraining (e.g., Audioset) but under moderate data remains less competitive. The results highlight a practical trade-off: lightweight stacked-CNNs are well-suited for edge and real-time deployments, whereas transformer-based models excel with abundant pretraining resources. These insights guide deployment decisions for ESC systems in constrained environments and point to future work on online/adaptive learning for dynamic acoustics.

Abstract

Environmental sound classification (ESC) has gained significant attention due to its diverse applications in smart city monitoring, fault detection, acoustic surveillance, and manufacturing quality control. To enhance CNN performance, feature stacking techniques have been explored to aggregate complementary acoustic descriptors into richer input representations. In this paper, we investigate CNN-based models employing various stacked feature combinations, including Log-Mel Spectrogram (LM), Spectral Contrast (SPC), Chroma (CH), Tonnetz (TZ), Mel-Frequency Cepstral Coefficients (MFCCs), and Gammatone Cepstral Coefficients (GTCC). Experiments are conducted on the widely used ESC-50 and UrbanSound8K datasets under different training regimes, including pretraining on ESC-50, fine-tuning on UrbanSound8K, and comparison with Audio Spectrogram Transformer (AST) models pretrained on large-scale corpora such as AudioSet. This experimental design enables an analysis of how feature-stacked CNNs compare with transformer-based models under varying levels of training data and pretraining diversity. The results indicate that feature-stacked CNNs offer a more computationally and data-efficient alternative when large-scale pretraining or extensive training data are unavailable, making them particularly well suited for resource-constrained and edge-level sound classification scenarios.

Performance Comparison of CNN and AST Models with Stacked Features for Environmental Sound Classification

TL;DR

The paper tackles environmental sound classification (ESC) under data- and resource-constrained conditions by evaluating CNNs trained on stacked feature representations (Log-Mel, MFCC, GTCC, Chroma, Spectral Contrast, Tonnetz) and comparing them to an Audio Spectrogram Transformer (AST) across ESC-50 and UrbanSound8K with varying pretraining. It demonstrates that feature-stacked CNNs offer data- and compute-efficient performance, with CNN-1 achieving up to 92.46% validation accuracy on ESC-50 when pre-trained on ESC-50 and fine-tuned on UrbanSound8K, while AST benefits substantially from large-scale pretraining (e.g., Audioset) but under moderate data remains less competitive. The results highlight a practical trade-off: lightweight stacked-CNNs are well-suited for edge and real-time deployments, whereas transformer-based models excel with abundant pretraining resources. These insights guide deployment decisions for ESC systems in constrained environments and point to future work on online/adaptive learning for dynamic acoustics.

Abstract

Environmental sound classification (ESC) has gained significant attention due to its diverse applications in smart city monitoring, fault detection, acoustic surveillance, and manufacturing quality control. To enhance CNN performance, feature stacking techniques have been explored to aggregate complementary acoustic descriptors into richer input representations. In this paper, we investigate CNN-based models employing various stacked feature combinations, including Log-Mel Spectrogram (LM), Spectral Contrast (SPC), Chroma (CH), Tonnetz (TZ), Mel-Frequency Cepstral Coefficients (MFCCs), and Gammatone Cepstral Coefficients (GTCC). Experiments are conducted on the widely used ESC-50 and UrbanSound8K datasets under different training regimes, including pretraining on ESC-50, fine-tuning on UrbanSound8K, and comparison with Audio Spectrogram Transformer (AST) models pretrained on large-scale corpora such as AudioSet. This experimental design enables an analysis of how feature-stacked CNNs compare with transformer-based models under varying levels of training data and pretraining diversity. The results indicate that feature-stacked CNNs offer a more computationally and data-efficient alternative when large-scale pretraining or extensive training data are unavailable, making them particularly well suited for resource-constrained and edge-level sound classification scenarios.
Paper Structure (17 sections, 1 figure, 3 tables)

This paper contains 17 sections, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Stacked Features
  5. Deep Learning Architectures
  6. Convolutional Neural Network (CNN) architectures
  7. CNN Model 1 (Baseline CNN):
  8. CNN Model 2 (Enhanced CNN with Batch Normalization and Dropout):
  9. Audio Spectrogram Transformer (AST) architecture
  10. Retraining the models using UrbanSound8K dataset
  11. Experimental Setup
  12. Results and Analysis
  13. Performance Trends and Observations
  14. Comparison with AST Model
  15. Training Time and Computational Efficiency Analysis
  16. ...and 2 more sections

Figures (1)

  • Figure 1: Comparison of training and validation accuracy and loss for CNN-1 and CNN-2 using stacked MFCC, Chroma, Mel-spectrogram, and GTCC features.