Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LibriVAD: A Scalable Open Dataset with Deep Learning Benchmarks for Voice Activity Detection

Ioannis Stylianou, Achintya kr. Sarkar, Nauman Dawalatabad, James Glass, Zheng-Hua Tan

TL;DR

LibriVAD addresses the shortage of large-scale, controllable VAD datasets by introducing a scalable, open-source benchmark derived from LibriSpeech, augmented with diverse real-world and synthetic noises and fine-grained SSR/SNR control. The authors benchmark multiple feature-model pairs and introduce a Vision Transformer-based VAD that, with MFCC features, consistently outperforms established baselines and generalizes to out-of-distribution data like VOiCES. They show that increasing dataset size and balancing SSR improve generalization, especially under reverberant and noisy conditions, and publicly release datasets, code, and trained models to foster reproducibility. The work advances robust VAD research and provides a practical, scalable platform for evaluating noise-robust VAD systems across seen, unseen, and real-world conditions.

Abstract

Robust Voice Activity Detection (VAD) remains a challenging task, especially under noisy, diverse, and unseen acoustic conditions. Beyond algorithmic development, a key limitation in advancing VAD research is the lack of large-scale, systematically controlled, and publicly available datasets. To address this, we introduce LibriVAD - a scalable open-source dataset derived from LibriSpeech and augmented with diverse real-world and synthetic noise sources. LibriVAD enables systematic control over speech-to-noise ratio, silence-to-speech ratio (SSR), and noise diversity, and is released in three sizes (15 GB, 150 GB, and 1.5 TB) with two variants (LibriVAD-NonConcat and LibriVAD-Concat) to support different experimental setups. We benchmark multiple feature-model combinations, including waveform, Mel-Frequency Cepstral Coefficients (MFCC), and Gammatone filter bank cepstral coefficients, and introduce the Vision Transformer (ViT) architecture for VAD. Our experiments show that ViT with MFCC features consistently outperforms established VAD models such as boosted deep neural network and convolutional long short-term memory deep neural network across seen, unseen, and out-of-distribution (OOD) conditions, including evaluation on the real-world VOiCES dataset. We further analyze the impact of dataset size and SSR on model generalization, experimentally showing that scaling up dataset size and balancing SSR noticeably and consistently enhance VAD performance under OOD conditions. All datasets, trained models, and code are publicly released to foster reproducibility and accelerate progress in VAD research.

LibriVAD: A Scalable Open Dataset with Deep Learning Benchmarks for Voice Activity Detection

TL;DR

LibriVAD addresses the shortage of large-scale, controllable VAD datasets by introducing a scalable, open-source benchmark derived from LibriSpeech, augmented with diverse real-world and synthetic noises and fine-grained SSR/SNR control. The authors benchmark multiple feature-model pairs and introduce a Vision Transformer-based VAD that, with MFCC features, consistently outperforms established baselines and generalizes to out-of-distribution data like VOiCES. They show that increasing dataset size and balancing SSR improve generalization, especially under reverberant and noisy conditions, and publicly release datasets, code, and trained models to foster reproducibility. The work advances robust VAD research and provides a practical, scalable platform for evaluating noise-robust VAD systems across seen, unseen, and real-world conditions.

Abstract

Robust Voice Activity Detection (VAD) remains a challenging task, especially under noisy, diverse, and unseen acoustic conditions. Beyond algorithmic development, a key limitation in advancing VAD research is the lack of large-scale, systematically controlled, and publicly available datasets. To address this, we introduce LibriVAD - a scalable open-source dataset derived from LibriSpeech and augmented with diverse real-world and synthetic noise sources. LibriVAD enables systematic control over speech-to-noise ratio, silence-to-speech ratio (SSR), and noise diversity, and is released in three sizes (15 GB, 150 GB, and 1.5 TB) with two variants (LibriVAD-NonConcat and LibriVAD-Concat) to support different experimental setups. We benchmark multiple feature-model combinations, including waveform, Mel-Frequency Cepstral Coefficients (MFCC), and Gammatone filter bank cepstral coefficients, and introduce the Vision Transformer (ViT) architecture for VAD. Our experiments show that ViT with MFCC features consistently outperforms established VAD models such as boosted deep neural network and convolutional long short-term memory deep neural network across seen, unseen, and out-of-distribution (OOD) conditions, including evaluation on the real-world VOiCES dataset. We further analyze the impact of dataset size and SSR on model generalization, experimentally showing that scaling up dataset size and balancing SSR noticeably and consistently enhance VAD performance under OOD conditions. All datasets, trained models, and code are publicly released to foster reproducibility and accelerate progress in VAD research.

Paper Structure

This paper contains 18 sections, 2 equations, 2 figures, 10 tables.

Table of Contents

  1. Introduction
  2. LibriVAD
  3. Silence Injection and Speech-to-Silence Ratio Adjustment
  4. Noise Selection and Generation
  5. LibriVAD Dataset Composition and Generation Process
  6. VAD Methods
  7. Features
  8. VAD Models
  9. Evaluation Metrics
  10. Experimental Setup
  11. Results and Discussions
  12. Analysis of VAD Performance on LibriVAD-NonConcat and LibriVAD-Concat
  13. Effect of Training Data Size on Generalization
  14. Cross-Dataset Evaluation of VAD Systems
  15. Performance of VAD Systems based on EER and MinDCF
  16. ...and 3 more sections

Figures (2)

  • Figure 1: An example of forced alignments on a clip from the Librispeech dataset. The bottom plot illustrates the phoneme boundaries, the top plot shows the corresponding word boundaries generated by the Montreal Forced Aligner, and the middle plot presents the spectrogram.
  • Figure 2: DET curves for different VAD models across a wide operating range, with scores aggregated over various noise types and SNR levels. (a) Model trained and evaluated on the small set of LibriVAD-NonConcat. (b) Model trained and evaluated on the small set of LibriVAD-Concat.