Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Comparison of Speech Data Augmentation Methods Using S3PRL Toolkit

Mina Huh, Ruchira Ray, Corey Karnei

TL;DR

This study uses the S3PRL toolkit to benchmark two state-of-the-art self-supervised speech representations, wav2vec and HuBERT, across phoneme recognition (PR) and automatic speech recognition (ASR) tasks. It compares three augmentation strategies—SpecAugment (feature-level), Gaussian Noise, and Speed Perturbation (data-level)—applied to LibriSpeech-based training data, and evaluates performance on both original and augmented test sets. SpecAugment generally preserves original performance, while Gaussian Noise and Speed Perturbation improve robustness to their corresponding augmented test conditions, with substantial PER and WER improvements when matched to the test conditions. The results underscore the value of task-specific augmentation for robustness, and point to future work involving sequential augmentations and broader dataset testing to generalize findings to real-world conditions.

Abstract

Data augmentations are known to improve robustness in speech-processing tasks. In this study, we summarize and compare different data augmentation strategies using S3PRL toolkit. We explore how HuBERT and wav2vec perform using different augmentation techniques (SpecAugment, Gaussian Noise, Speed Perturbation) for Phoneme Recognition (PR) and Automatic Speech Recognition (ASR) tasks. We evaluate model performance in terms of phoneme error rate (PER) and word error rate (WER). From the experiments, we observed that SpecAugment slightly improves the performance of HuBERT and wav2vec on the original dataset. Also, we show that models trained using the Gaussian Noise and Speed Perturbation dataset are more robust when tested with augmented test sets.

A Comparison of Speech Data Augmentation Methods Using S3PRL Toolkit

TL;DR

This study uses the S3PRL toolkit to benchmark two state-of-the-art self-supervised speech representations, wav2vec and HuBERT, across phoneme recognition (PR) and automatic speech recognition (ASR) tasks. It compares three augmentation strategies—SpecAugment (feature-level), Gaussian Noise, and Speed Perturbation (data-level)—applied to LibriSpeech-based training data, and evaluates performance on both original and augmented test sets. SpecAugment generally preserves original performance, while Gaussian Noise and Speed Perturbation improve robustness to their corresponding augmented test conditions, with substantial PER and WER improvements when matched to the test conditions. The results underscore the value of task-specific augmentation for robustness, and point to future work involving sequential augmentations and broader dataset testing to generalize findings to real-world conditions.

Abstract

Data augmentations are known to improve robustness in speech-processing tasks. In this study, we summarize and compare different data augmentation strategies using S3PRL toolkit. We explore how HuBERT and wav2vec perform using different augmentation techniques (SpecAugment, Gaussian Noise, Speed Perturbation) for Phoneme Recognition (PR) and Automatic Speech Recognition (ASR) tasks. We evaluate model performance in terms of phoneme error rate (PER) and word error rate (WER). From the experiments, we observed that SpecAugment slightly improves the performance of HuBERT and wav2vec on the original dataset. Also, we show that models trained using the Gaussian Noise and Speed Perturbation dataset are more robust when tested with augmented test sets.
Paper Structure (17 sections, 2 figures, 2 tables)

This paper contains 17 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. S3PRL
  4. Augmentation Methods
  5. Data
  6. Dataset
  7. Data Preparation
  8. Data Augmentation
  9. Experiments
  10. Speech-Processing Tasks
  11. Self-supervised Pre-trained Models
  12. wav2vec
  13. HuBERT
  14. Experimental Setup
  15. Results
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Examples of spectrograms after data augmentation (Speed Perturbation and Gaussian Noise)((Fig \ref{['fig:sub-third']}, \ref{['fig:sub-fourth']}) is applied and feature augmentation (Fig \ref{['fig:sub-second']}) are applied separately. The original audio spectrogram is shown in the first spectrogram (Fig \ref{['fig:sub-first']})
  • Figure 2: We selected the 100-hour dataset from the LibriSpeech corpus and applied our chosen augmentations to it to generate an augmented dataset. For the Phoneme recognition task, the SpecAugment dataset and Gaussian Noise dataset were used. For the Automatic speech recognition task, the SpecAugment dataset and Speed Perturbation dataset were used.