Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fitting Auditory Filterbanks with Multiresolution Neural Networks

Vincent Lostanlen, Daniel Haider, Han Han, Mathieu Lagrange, Peter Balazs, Martin Ehler

TL;DR

This paper introduces a neural audio model, named multiresolution neural network (MuReNN), to train separate convolutional operators over the octave subbands of a discrete wavelet transform (DWT), and finds that MuReNN reaches state-of-the-art performance on all three optimization problems.

Abstract

Waveform-based deep learning faces a dilemma between nonparametric and parametric approaches. On one hand, convolutional neural networks (convnets) may approximate any linear time-invariant system; yet, in practice, their frequency responses become more irregular as their receptive fields grow. On the other hand, a parametric model such as LEAF is guaranteed to yield Gabor filters, hence an optimal time-frequency localization; yet, this strong inductive bias comes at the detriment of representational capacity. In this paper, we aim to overcome this dilemma by introducing a neural audio model, named multiresolution neural network (MuReNN). The key idea behind MuReNN is to train separate convolutional operators over the octave subbands of a discrete wavelet transform (DWT). Since the scale of DWT atoms grows exponentially between octaves, the receptive fields of the subsequent learnable convolutions in MuReNN are dilated accordingly. For a given real-world dataset, we fit the magnitude response of MuReNN to that of a well-established auditory filterbank: Gammatone for speech, CQT for music, and third-octave for urban sounds, respectively. This is a form of knowledge distillation (KD), in which the filterbank ''teacher'' is engineered by domain knowledge while the neural network ''student'' is optimized from data. We compare MuReNN to the state of the art in terms of goodness of fit after KD on a hold-out set and in terms of Heisenberg time-frequency localization. Compared to convnets and Gabor convolutions, we find that MuReNN reaches state-of-the-art performance on all three optimization problems.

Fitting Auditory Filterbanks with Multiresolution Neural Networks

TL;DR

This paper introduces a neural audio model, named multiresolution neural network (MuReNN), to train separate convolutional operators over the octave subbands of a discrete wavelet transform (DWT), and finds that MuReNN reaches state-of-the-art performance on all three optimization problems.

Abstract

Waveform-based deep learning faces a dilemma between nonparametric and parametric approaches. On one hand, convolutional neural networks (convnets) may approximate any linear time-invariant system; yet, in practice, their frequency responses become more irregular as their receptive fields grow. On the other hand, a parametric model such as LEAF is guaranteed to yield Gabor filters, hence an optimal time-frequency localization; yet, this strong inductive bias comes at the detriment of representational capacity. In this paper, we aim to overcome this dilemma by introducing a neural audio model, named multiresolution neural network (MuReNN). The key idea behind MuReNN is to train separate convolutional operators over the octave subbands of a discrete wavelet transform (DWT). Since the scale of DWT atoms grows exponentially between octaves, the receptive fields of the subsequent learnable convolutions in MuReNN are dilated accordingly. For a given real-world dataset, we fit the magnitude response of MuReNN to that of a well-established auditory filterbank: Gammatone for speech, CQT for music, and third-octave for urban sounds, respectively. This is a form of knowledge distillation (KD), in which the filterbank ''teacher'' is engineered by domain knowledge while the neural network ''student'' is optimized from data. We compare MuReNN to the state of the art in terms of goodness of fit after KD on a hold-out set and in terms of Heisenberg time-frequency localization. Compared to convnets and Gabor convolutions, we find that MuReNN reaches state-of-the-art performance on all three optimization problems.
Paper Structure (14 sections, 6 equations, 4 figures, 1 table)

This paper contains 14 sections, 6 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Neural audio models
  3. Learnable time-domain filterbanks (Conv1D)
  4. Gabor 1-D convolutions (Gabor1D)
  5. Multiresolution neural network (MuReNN)
  6. Knowledge distillation
  7. Target auditory filterbanks
  8. Gradient-based optimization
  9. Results and discussion
  10. Datasets
  11. Benchmarks
  12. Error analysis
  13. Conclusion
  14. ACKNOWLEDGMENT

Figures (4)

  • Figure 1: Graphical outline of the proposed method. We train a neural network "student" $\boldsymbol{\Phi}_{\mathbf{W}}$ to regress the squared magnitudes $\mathbf{Y}$ of an auditory filterbank "teacher" $\boldsymbol{\Lambda}$ in terms of spectrogram-based cosine distance $\mathcal{L}_{\mathbf{x}}$, on average over a dataset of natural sounds $\boldsymbol{x}$.
  • Figure 2: Left to right: evolution of validation losses on different domains with Conv1D (green), Gabor1D (blue), and MuReNN (orange), as a function of training epochs. The shaded area denotes the standard deviation across five independent trials. See Section \ref{['sub:benchmark']} for details.
  • Figure 3: Compared impulse responses of Conv1D (left), Gabor1D (center), and MuReNN (right) with different center frequencies after convergence, with a Gammatone filterbank as target. Solid blue (resp. dashed red) lines denote the real part of the impulse responses of the learned filters (resp. target). See Section \ref{['sub:qualitative']} for details.
  • Figure 4: Distribution of Heisenberg time--frequency ratios for each teacher--student pair (lower is better). See Section \ref{['sub:qualitative']} for details.