ChildAugment: Data Augmentation Methods for Zero-Resource Children's Speaker Verification

TL;DR

The findings on the CSLU kids corpus indicate that ChildAugment holds promise as a simple, acoustics-motivated approach, for improving state-of-the-art deep learning based ASV for children.

Abstract

The accuracy of modern automatic speaker verification (ASV) systems, when trained exclusively on adult data, drops substantially when applied to children's speech. The scarcity of children's speech corpora hinders fine-tuning ASV systems for children's speech. Hence, there is a timely need to explore more effective ways of reusing adults' speech data. One promising approach is to align vocal-tract parameters between adults and children through children-specific data augmentation, referred here to as ChildAugment. Specifically, we modify the formant frequencies and formant bandwidths of adult speech to emulate children's speech. The modified spectra are used to train ECAPA-TDNN (emphasized channel attention, propagation, and aggregation in time-delay neural network) recognizer for children. We compare ChildAugment against various state-of-the-art data augmentation techniques for children's ASV. We also extensively compare different scoring methods, including cosine scoring, PLDA (probabilistic linear discriminant analysis), and NPLDA (neural PLDA). We also propose a low-complexity weighted cosine score for extremely low-resource children ASV. Our findings on the CSLU kids corpus indicate that ChildAugment holds promise as a simple, acoustics-motivated approach, for improving state-of-the-art deep learning based ASV for children. We achieve up to 12.45% (boys) and 11.96% (girls) relative improvement over the baseline.

Figures (4)

• Figure 1: Data augmentation methods for training speaker embedding extractor. Original data to augmentation ratio and relative weight of each augmentation method are also defined. Here, $\gamma_{i}$ is the weight of a particular augmentation method, $\gamma_{0}$ is the weight of original data, $N$ is the batch size, and $M$ is the number of augmentation methods. We present the augmentation methods and their weights ($\gamma_{i}$) in Table \ref{['tab:res2']}. While VTLP and LPC-WP were previously explored for children ASR in LPCAugmentkathania_aug, we have investigated their impact on children's ASV tasks. Our work introduces a novel and more systematic algorithm for utilizing the subsequent method highlighted with shaded background, previously explored for children's ASR in me2022, for children's ASV tasks. The second last method in the shaded background is novel and proposed in this paper.
• Figure 2: (a) Acoustically motivated children-specific data augmentation pipeline. Shaded blocks indicate the key changes included for LPC-SWP and BWP-FEP modifications. $\alpha_k$ and $\beta_k$ are pole's phase warping and radius scaling factors for a set of poles responsible for forming $k$-th formant, respectively; (b) Visualization of the original LPC roots for input frame, and corresponding LPC-SWP and BWP-FEP modified roots for output frames in the z-domain unit circle, for vowel /a/; and (c) LPC spectra of an adult speech (input frame), LPC-SWP and BWP-FEP based modified adult spectra (output frames), along with the children spectra for a vowel /a/ computed in 25 ms window. Different LPC spectra are separated by 30 dB on the y-axis for better visualization.
• Figure 3: Block diagram representing the ECAP-TDNN training in zero-resource, and PLDA/NPLDA/W-Cosine training in low-resource scenarios.
• Figure 4: Comparison of scoring methods in different low resource conditions. Results are presented for Proposed-$3/11$ model on S2S evaluation set of CSLU boys speakers. (1) entire Dev-good dataset (total 6.13 Hours), (2) 3 Hours from Dev-good dataset, (3) 1.5 Hours from Dev-good dataset, (4) 36 minutes from Dev-good dataset. In all, 4 cases selected utterances were equally distributed across all speakers. The exact number of parameters in different scoring methods are also included.