Enhancing Synthetic Training Data for Speech Commands: From ASR-Based Filtering to Domain Adaptation in SSL Latent Space

TL;DR

The results on the Google Speech Commands dataset show that a simple ASR-based filtering method can have a big impact in the quality of the generated data, translating to a better performance, and show that synthetic and real speech can still be easily distinguishable when using self-supervised (WavLM) features.

Abstract

The use of synthetic speech as data augmentation is gaining increasing popularity in fields such as automatic speech recognition and speech classification tasks. Despite novel text-to-speech systems with voice cloning capabilities, that allow the usage of a larger amount of voices based on short audio segments, it is known that these systems tend to hallucinate and oftentimes produce bad data that will most likely have a negative impact on the downstream task. In the present work, we conduct a set of experiments around zero-shot learning with synthetic speech data for the specific task of speech commands classification. Our results on the Google Speech Commands dataset show that a simple ASR-based filtering method can have a big impact in the quality of the generated data, translating to a better performance. Furthermore, despite the good quality of the generated speech data, we also show that synthetic and real speech can still be easily distinguishable when using self-supervised (WavLM) features, an aspect further explored with a CycleGAN to bridge the gap between the two types of speech material.

Figures (3)

• Figure 1: Proposed TTS generation loop with ASR filtering.
• Figure 2: PCA analysis of filtered synthetic data and real speech data using Mel Frequency Cesptral Coefficients (MFCC) features (left) and WavLM self-supervised feature (right).
• Figure 3: Proposed CycleGAN architecture and domain adaptation for a WavLM based SCC system.