Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

sVAD: A Robust, Low-Power, and Light-Weight Voice Activity Detection with Spiking Neural Networks

Qu Yang, Qianhui Liu, Nan Li, Meng Ge, Zeyang Song, Haizhou Li

TL;DR

This work tackles the challenge of robust, energy-efficient voice activity detection by introducing sVAD, an SNN-based VAD with an auditory encoder that incorporates an SNN-based attention mechanism and a spiking recurrent neural network classifier. The auditory pathway leverages SincNet with a data-driven 1D convolution to produce spike-based features, while attention enhances robustness in adverse acoustics. The model jointly optimizes a classification loss and an attention mask loss, trained with Backpropagation Through Time using a surrogate gradient. Results on the QUT-NOISE-TIMIT corpus show strong noise robustness, low latency, and a small footprint, with energy estimates on the Loihi chip confirming favorable power characteristics for real-world deployment.

Abstract

Speech applications are expected to be low-power and robust under noisy conditions. An effective Voice Activity Detection (VAD) front-end lowers the computational need. Spiking Neural Networks (SNNs) are known to be biologically plausible and power-efficient. However, SNN-based VADs have yet to achieve noise robustness and often require large models for high performance. This paper introduces a novel SNN-based VAD model, referred to as sVAD, which features an auditory encoder with an SNN-based attention mechanism. Particularly, it provides effective auditory feature representation through SincNet and 1D convolution, and improves noise robustness with attention mechanisms. The classifier utilizes Spiking Recurrent Neural Networks (sRNN) to exploit temporal speech information. Experimental results demonstrate that our sVAD achieves remarkable noise robustness and meanwhile maintains low power consumption and a small footprint, making it a promising solution for real-world VAD applications.

sVAD: A Robust, Low-Power, and Light-Weight Voice Activity Detection with Spiking Neural Networks

TL;DR

This work tackles the challenge of robust, energy-efficient voice activity detection by introducing sVAD, an SNN-based VAD with an auditory encoder that incorporates an SNN-based attention mechanism and a spiking recurrent neural network classifier. The auditory pathway leverages SincNet with a data-driven 1D convolution to produce spike-based features, while attention enhances robustness in adverse acoustics. The model jointly optimizes a classification loss and an attention mask loss, trained with Backpropagation Through Time using a surrogate gradient. Results on the QUT-NOISE-TIMIT corpus show strong noise robustness, low latency, and a small footprint, with energy estimates on the Loihi chip confirming favorable power characteristics for real-world deployment.

Abstract

Speech applications are expected to be low-power and robust under noisy conditions. An effective Voice Activity Detection (VAD) front-end lowers the computational need. Spiking Neural Networks (SNNs) are known to be biologically plausible and power-efficient. However, SNN-based VADs have yet to achieve noise robustness and often require large models for high performance. This paper introduces a novel SNN-based VAD model, referred to as sVAD, which features an auditory encoder with an SNN-based attention mechanism. Particularly, it provides effective auditory feature representation through SincNet and 1D convolution, and improves noise robustness with attention mechanisms. The classifier utilizes Spiking Recurrent Neural Networks (sRNN) to exploit temporal speech information. Experimental results demonstrate that our sVAD achieves remarkable noise robustness and meanwhile maintains low power consumption and a small footprint, making it a promising solution for real-world VAD applications.
Paper Structure (17 sections, 9 equations, 2 figures, 3 tables)

This paper contains 17 sections, 9 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. SNN-based VAD with Auditory Attention
  3. Spiking Neuron Model
  4. Auditory Encoder with Attention
  5. Auditory encoder
  6. Attention mechanism
  7. Model Learning
  8. Spiking recurrent neural network for classification
  9. Loss function
  10. SNN training algorithm
  11. Experiments and Results
  12. Dataset and setup
  13. Results and analysis
  14. Compare with existing models
  15. Ablation study
  16. ...and 2 more sections

Figures (2)

  • Figure 1: The proposed SNN-based VAD model consists of an auditory encoder with attention for feature extraction and a classifier for frame-level classification.
  • Figure 2: Comparisons of HTER performance for different noise levels. Dark and light shadings denote the miss rate and false alarm rate contributions, respectively.