Adversarial Data Augmentation for Robust Speaker Verification

TL;DR

The paper addresses robustness gaps in ASV caused by diverse acoustic variations and augmentation residual in vanilla DA. It proposes Adversarial Data Augmentation (A-DA), which pairs standard DA with an augmentation classifier and a gradient reversal layer to encourage speaker embeddings to be invariant to augmentation types. The learning objective combines $L = L_{spk} + \lambda L_{adv}$, with $L_{spk}$ using AAM-Softmax and $L_{adv}$ as binary cross-entropy; $\lambda$ is set to 0.01. Experiments on VoxCeleb and CN-Celeb show A-DA consistently outperforms vanilla DA, especially under unseen augmentations and multi-genre conditions, demonstrating improved robustness and generalization.

Abstract

Data augmentation (DA) has gained widespread popularity in deep speaker models due to its ease of implementation and significant effectiveness. It enriches training data by simulating real-life acoustic variations, enabling deep neural networks to learn speaker-related representations while disregarding irrelevant acoustic variations, thereby improving robustness and generalization. However, a potential issue with the vanilla DA is augmentation residual, i.e., unwanted distortion caused by different types of augmentation. To address this problem, this paper proposes a novel approach called adversarial data augmentation (A-DA) which combines DA with adversarial learning. Specifically, it involves an additional augmentation classifier to categorize various augmentation types used in data augmentation. This adversarial learning empowers the network to generate speaker embeddings that can deceive the augmentation classifier, making the learned speaker embeddings more robust in the face of augmentation variations. Experiments conducted on VoxCeleb and CN-Celeb datasets demonstrate that our proposed A-DA outperforms standard DA in both augmentation matched and mismatched test conditions, showcasing its superior robustness and generalization against acoustic variations.

Figures (1)

• Figure 1: Illustration on the training strategy of our proposed adversarial data augmentation method.