Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multitaper mel-spectrograms for keyword spotting

Douglas Baptista de Souza, Khaled Jamal Bakri, Fernanda Ferreira, Juliana Inacio

TL;DR

The paper tackles the sensitivity of keyword spotting performance to feature representation on resource-constrained devices. It introduces multitaper-mel spectrograms, deriving the multitaper spectrum estimator $\widehat{S}_{w}^{(K)}(\tau,f)$ and the corresponding mel-domain feature $\hat{\mathbf{s}}^{\text{mel}}_{K,\tau}$ to reduce variance in spectral estimates. Evaluations on two small-footprint networks, Tiny-CNN and TC-ResNet, across Google Speech Commands v2 and Mozilla Spoken Digits under noisy conditions demonstrate consistent performance gains, with inference time increasing roughly linearly with $K$ and with SWCE-based tapers often delivering the strongest improvements. The results support deploying multitaper-based features in embedded KWS systems and point to future work on automatic learning of KWS feature representations.

Abstract

Keyword spotting (KWS) is one of the speech recognition tasks most sensitive to the quality of the feature representation. However, the research on KWS has traditionally focused on new model topologies, putting little emphasis on other aspects like feature extraction. This paper investigates the use of the multitaper technique to create improved features for KWS. The experimental study is carried out for different test scenarios, windows and parameters, datasets, and neural networks commonly used in embedded KWS applications. Experiment results confirm the advantages of using the proposed improved features.

Multitaper mel-spectrograms for keyword spotting

TL;DR

The paper tackles the sensitivity of keyword spotting performance to feature representation on resource-constrained devices. It introduces multitaper-mel spectrograms, deriving the multitaper spectrum estimator and the corresponding mel-domain feature to reduce variance in spectral estimates. Evaluations on two small-footprint networks, Tiny-CNN and TC-ResNet, across Google Speech Commands v2 and Mozilla Spoken Digits under noisy conditions demonstrate consistent performance gains, with inference time increasing roughly linearly with and with SWCE-based tapers often delivering the strongest improvements. The results support deploying multitaper-based features in embedded KWS systems and point to future work on automatic learning of KWS feature representations.

Abstract

Keyword spotting (KWS) is one of the speech recognition tasks most sensitive to the quality of the feature representation. However, the research on KWS has traditionally focused on new model topologies, putting little emphasis on other aspects like feature extraction. This paper investigates the use of the multitaper technique to create improved features for KWS. The experimental study is carried out for different test scenarios, windows and parameters, datasets, and neural networks commonly used in embedded KWS applications. Experiment results confirm the advantages of using the proposed improved features.
Paper Structure (13 sections, 8 equations, 2 figures, 2 tables)

This paper contains 13 sections, 8 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background elements
  3. The multitaper technique
  4. Multitaper-mel spectrograms
  5. Multitaper windows
  6. Hermite window functions
  7. SWCE window functions
  8. Experimental study
  9. Chosen KWS architectures
  10. Datasets
  11. Experiment design choices
  12. Test results
  13. Conclusions

Figures (2)

  • Figure 1: Regular spectrogram for the default Hann window (a), and multitaper ones with (b) Hermite, (c) SWCE, and (d) modified SWCE tapers.
  • Figure 2: For the TC-ResNet model, average inference time vs. $K$ by using different multitaper-mel spectrogram features. For the simulations, "yes" audios samples from the Google Speech Commands dataset have been tested.