Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving Machine Hearing on Limited Data Sets

Pavol Harar, Roswitha Bammer, Anna Breger, Monika Dörfler, Zdenek Smekal

TL;DR

The paper addresses data efficiency in audio classification with CNNs by replacing standard mel-spectrogram inputs with Mel Scattering (MS) and by using an Augmented Target loss (AT) to incorporate additional target information. It introduces and evaluates MS and Gabor Scattering (GS) representations, and demonstrates that these time-frequency representations, together with AT, yield superior performance over conventional mel-spectrogram inputs when training data are scarce. Using the GoodSounds instrument dataset, the study shows that MS and GS improve accuracy and speed of learning in low-data regimes, while AT provides targeted improvements with modest training cost. The findings highlight the potential of multi-resolution time-frequency representations and informative target transformations to enhance data-efficient learning in music information retrieval, with reproducible code and experiments for further exploration.

Abstract

Convolutional neural network (CNN) architectures have originated and revolutionized machine learning for images. In order to take advantage of CNNs in predictive modeling with audio data, standard FFT-based signal processing methods are often applied to convert the raw audio waveforms into an image-like representations (e.g. spectrograms). Even though conventional images and spectrograms differ in their feature properties, this kind of pre-processing reduces the amount of training data necessary for successful training. In this contribution we investigate how input and target representations interplay with the amount of available training data in a music information retrieval setting. We compare the standard mel-spectrogram inputs with a newly proposed representation, called Mel scattering. Furthermore, we investigate the impact of additional target data representations by using an augmented target loss function which incorporates unused available information. We observe that all proposed methods outperform the standard mel-transform representation when using a limited data set and discuss their strengths and limitations. The source code for reproducibility of our experiments as well as intermediate results and model checkpoints are available in an online repository.

Improving Machine Hearing on Limited Data Sets

TL;DR

The paper addresses data efficiency in audio classification with CNNs by replacing standard mel-spectrogram inputs with Mel Scattering (MS) and by using an Augmented Target loss (AT) to incorporate additional target information. It introduces and evaluates MS and Gabor Scattering (GS) representations, and demonstrates that these time-frequency representations, together with AT, yield superior performance over conventional mel-spectrogram inputs when training data are scarce. Using the GoodSounds instrument dataset, the study shows that MS and GS improve accuracy and speed of learning in low-data regimes, while AT provides targeted improvements with modest training cost. The findings highlight the potential of multi-resolution time-frequency representations and informative target transformations to enhance data-efficient learning in music information retrieval, with reproducible code and experiments for further exploration.

Abstract

Convolutional neural network (CNN) architectures have originated and revolutionized machine learning for images. In order to take advantage of CNNs in predictive modeling with audio data, standard FFT-based signal processing methods are often applied to convert the raw audio waveforms into an image-like representations (e.g. spectrograms). Even though conventional images and spectrograms differ in their feature properties, this kind of pre-processing reduces the amount of training data necessary for successful training. In this contribution we investigate how input and target representations interplay with the amount of available training data in a music information retrieval setting. We compare the standard mel-spectrogram inputs with a newly proposed representation, called Mel scattering. Furthermore, we investigate the impact of additional target data representations by using an augmented target loss function which incorporates unused available information. We observe that all proposed methods outperform the standard mel-transform representation when using a limited data set and discuss their strengths and limitations. The source code for reproducibility of our experiments as well as intermediate results and model checkpoints are available in an online repository.

Paper Structure

This paper contains 13 sections, 10 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Learning from Data
  3. Data Set used for Experiments
  4. Time-Frequency Representations of Audio
  5. Spectrograms and Mel-Spectrograms
  6. Gabor Scattering and Mel Scattering
  7. Augmented Target Loss Function
  8. Design of Transformations
  9. Numerical Experiments
  10. Computation of Signal Representations
  11. Deep Convolutional Neural Network
  12. Training and Results
  13. Discussion and Conclusions

Figures (3)

  • Figure 1: Visualization of time-frequency transformations.
  • Figure 2: CNN performance milestone reached over number of weight updates. The computational effort in all experiments was limited to 11 000 weight updates. Figure notation: Valid acc -- Accuracy performance metric measured on the validation set, Best w.u. -- Weight update after which the highest performance was reached.
  • Figure 3: Visualization of occlusion maps and frequency channel importance based on the best performing model trained on 1 batch of MS. Signal shown is randomly selected alto sax sample. Figure notation: Input -- input representation for CNN. Occ -- occlusion map created by sliding occlusion window. Input $\odot$ Occ$^+$ -- Elementwise multiplication of input with positive semidefinite occ (negative elements were changed to zeros before multiplication). Blue and red colors -- Positive and negative influence of particular frequency channel bin on the model performance.