Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generalizable Audio Deepfake Detection via Latent Space Refinement and Augmentation

Wen Huang, Yanmei Gu, Zhiming Wang, Huijia Zhu, Yanmin Qian

TL;DR

A novel strategy that integrates Latent Space Refinement (LSR) and Latent Space Augmentation (LSA) to improve the generalization of deepfake detection systems and achieves competitive results, matching or surpassing current state-of-the-art methods.

Abstract

Advances in speech synthesis technologies, like text-to-speech (TTS) and voice conversion (VC), have made detecting deepfake speech increasingly challenging. Spoofing countermeasures often struggle to generalize effectively, particularly when faced with unseen attacks. To address this, we propose a novel strategy that integrates Latent Space Refinement (LSR) and Latent Space Augmentation (LSA) to improve the generalization of deepfake detection systems. LSR introduces multiple learnable prototypes for the spoof class, refining the latent space to better capture the intricate variations within spoofed data. LSA further diversifies spoofed data representations by applying augmentation techniques directly in the latent space, enabling the model to learn a broader range of spoofing patterns. We evaluated our approach on four representative datasets, i.e. ASVspoof 2019 LA, ASVspoof 2021 LA and DF, and In-The-Wild. The results show that LSR and LSA perform well individually, and their integration achieves competitive results, matching or surpassing current state-of-the-art methods.

Generalizable Audio Deepfake Detection via Latent Space Refinement and Augmentation

TL;DR

A novel strategy that integrates Latent Space Refinement (LSR) and Latent Space Augmentation (LSA) to improve the generalization of deepfake detection systems and achieves competitive results, matching or surpassing current state-of-the-art methods.

Abstract

Advances in speech synthesis technologies, like text-to-speech (TTS) and voice conversion (VC), have made detecting deepfake speech increasingly challenging. Spoofing countermeasures often struggle to generalize effectively, particularly when faced with unseen attacks. To address this, we propose a novel strategy that integrates Latent Space Refinement (LSR) and Latent Space Augmentation (LSA) to improve the generalization of deepfake detection systems. LSR introduces multiple learnable prototypes for the spoof class, refining the latent space to better capture the intricate variations within spoofed data. LSA further diversifies spoofed data representations by applying augmentation techniques directly in the latent space, enabling the model to learn a broader range of spoofing patterns. We evaluated our approach on four representative datasets, i.e. ASVspoof 2019 LA, ASVspoof 2021 LA and DF, and In-The-Wild. The results show that LSR and LSA perform well individually, and their integration achieves competitive results, matching or surpassing current state-of-the-art methods.
Paper Structure (10 sections, 10 equations, 3 figures, 4 tables)

This paper contains 10 sections, 10 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Latent Space Refinement
  4. Latent Space Augmentation
  5. Experiments
  6. Experimental Settings
  7. Overall Performance Comparison
  8. Ablation Study on Latent Space Refinement
  9. Ablation Study on Latent Space Augmentation
  10. Conclusions

Figures (3)

  • Figure 1: The pipeline of the proposed method, illustrating the process of Latent Space Refinement (LSR) and Latent Space Augmentation (LSA).
  • Figure 2: t-SNE visualization of the training dataset featuring various latent space augmentations. The green, blue, and red points represent the 2D projections of embeddings for the bonafide, spoof, and augmented spoof classes, respectively.
  • Figure 3: The effect of the number of spoofed prototypes on EER (%) across different datasets (21LA, 21DF, and ITW).