Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cross-Domain Audio Deepfake Detection: Dataset and Analysis

Yuang Li, Min Zhang, Mengxin Ren, Miaomiao Ma, Daimeng Wei, Hao Yang

TL;DR

A new cross-domain ADD dataset comprising over 300 hours of speech data that is generated by five advanced zero-shot TTS models is constructed, demonstrating the models’ outstanding few-shot ADD ability by fine-tuning with just one minute of target-domain data.

Abstract

Audio deepfake detection (ADD) is essential for preventing the misuse of synthetic voices that may infringe on personal rights and privacy. Recent zero-shot text-to-speech (TTS) models pose higher risks as they can clone voices with a single utterance. However, the existing ADD datasets are outdated, leading to suboptimal generalization of detection models. In this paper, we construct a new cross-domain ADD dataset comprising over 300 hours of speech data that is generated by five advanced zero-shot TTS models. To simulate real-world scenarios, we employ diverse attack methods and audio prompts from different datasets. Experiments show that, through novel attack-augmented training, the Wav2Vec2-large and Whisper-medium models achieve equal error rates of 4.1\% and 6.5\% respectively. Additionally, we demonstrate our models' outstanding few-shot ADD ability by fine-tuning with just one minute of target-domain data. Nonetheless, neural codec compressors greatly affect the detection accuracy, necessitating further research.

Cross-Domain Audio Deepfake Detection: Dataset and Analysis

TL;DR

A new cross-domain ADD dataset comprising over 300 hours of speech data that is generated by five advanced zero-shot TTS models is constructed, demonstrating the models’ outstanding few-shot ADD ability by fine-tuning with just one minute of target-domain data.

Abstract

Audio deepfake detection (ADD) is essential for preventing the misuse of synthetic voices that may infringe on personal rights and privacy. Recent zero-shot text-to-speech (TTS) models pose higher risks as they can clone voices with a single utterance. However, the existing ADD datasets are outdated, leading to suboptimal generalization of detection models. In this paper, we construct a new cross-domain ADD dataset comprising over 300 hours of speech data that is generated by five advanced zero-shot TTS models. To simulate real-world scenarios, we employ diverse attack methods and audio prompts from different datasets. Experiments show that, through novel attack-augmented training, the Wav2Vec2-large and Whisper-medium models achieve equal error rates of 4.1\% and 6.5\% respectively. Additionally, we demonstrate our models' outstanding few-shot ADD ability by fine-tuning with just one minute of target-domain data. Nonetheless, neural codec compressors greatly affect the detection accuracy, necessitating further research.
Paper Structure (14 sections, 4 figures, 4 tables, 1 algorithm)

This paper contains 14 sections, 4 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methods
  3. Dataset Construction
  4. Attacks
  5. ADD Methods
  6. Experimental Setups
  7. Experimental Results
  8. Pairwise Cross-Model Evaluation
  9. Comparisons Between Attacks
  10. Results of Few-Shot Fine-Tuning
  11. Conclusion
  12. Limitation
  13. Appendix: Open Source Tools
  14. Appendix: CD-ADD Dataset

Figures (4)

  • Figure 1: Zero-shot TTS architectures. a) Decoder-only. b) Encoder-decoder.
  • Figure 2: Categories of tested attacks.
  • Figure 3: Cross-model EER matrix, where the Wav2Vec2-base model was trained using data generated from a single TTS model and subsequently evaluated on data originating from other TTS models.
  • Figure 4: Few-shot performance of three base models measured by EER (%).