Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DFADD: The Diffusion and Flow-Matching Based Audio Deepfake Dataset

Jiawei Du, I-Ming Lin, I-Hsiang Chiu, Xuanjun Chen, Haibin Wu, Wenze Ren, Yu Tsao, Hung-yi Lee, Jyh-Shing Roger Jang

TL;DR

This work addresses the security risks posed by advanced diffusion- and flow-matching-based TTS deepfakes to anti-spoofing systems. It introduces DFADD, a large, speaker-disjoint dataset built from 109 VCTK speakers and five backbones (three diffusion-based: Grad-TTS, NaturalSpeech 2, Style-TTS 2; two FM-based: Matcha-TTS, PFlow-TTS) to simulate spoofed audio. Through extensive evaluations using a state-of-the-art anti-spoofing model (AASIST-L), the authors demonstrate that models trained on ASVspoof data struggle against DFADD-generated spoofs, while training on DFADD substantially improves robustness, including a reduction of average EER on unseen data by up to over 47%. The study highlights the importance of diffusion/FM-aware defenses and provides a strong resources for developing more resilient anti-spoofing models, with plans to release code and data.

Abstract

Mainstream zero-shot TTS production systems like Voicebox and Seed-TTS achieve human parity speech by leveraging Flow-matching and Diffusion models, respectively. Unfortunately, human-level audio synthesis leads to identity misuse and information security issues. Currently, many antispoofing models have been developed against deepfake audio. However, the efficacy of current state-of-the-art anti-spoofing models in countering audio synthesized by diffusion and flowmatching based TTS systems remains unknown. In this paper, we proposed the Diffusion and Flow-matching based Audio Deepfake (DFADD) dataset. The DFADD dataset collected the deepfake audio based on advanced diffusion and flowmatching TTS models. Additionally, we reveal that current anti-spoofing models lack sufficient robustness against highly human-like audio generated by diffusion and flow-matching TTS systems. The proposed DFADD dataset addresses this gap and provides a valuable resource for developing more resilient anti-spoofing models.

DFADD: The Diffusion and Flow-Matching Based Audio Deepfake Dataset

TL;DR

This work addresses the security risks posed by advanced diffusion- and flow-matching-based TTS deepfakes to anti-spoofing systems. It introduces DFADD, a large, speaker-disjoint dataset built from 109 VCTK speakers and five backbones (three diffusion-based: Grad-TTS, NaturalSpeech 2, Style-TTS 2; two FM-based: Matcha-TTS, PFlow-TTS) to simulate spoofed audio. Through extensive evaluations using a state-of-the-art anti-spoofing model (AASIST-L), the authors demonstrate that models trained on ASVspoof data struggle against DFADD-generated spoofs, while training on DFADD substantially improves robustness, including a reduction of average EER on unseen data by up to over 47%. The study highlights the importance of diffusion/FM-aware defenses and provides a strong resources for developing more resilient anti-spoofing models, with plans to release code and data.

Abstract

Mainstream zero-shot TTS production systems like Voicebox and Seed-TTS achieve human parity speech by leveraging Flow-matching and Diffusion models, respectively. Unfortunately, human-level audio synthesis leads to identity misuse and information security issues. Currently, many antispoofing models have been developed against deepfake audio. However, the efficacy of current state-of-the-art anti-spoofing models in countering audio synthesized by diffusion and flowmatching based TTS systems remains unknown. In this paper, we proposed the Diffusion and Flow-matching based Audio Deepfake (DFADD) dataset. The DFADD dataset collected the deepfake audio based on advanced diffusion and flowmatching TTS models. Additionally, we reveal that current anti-spoofing models lack sufficient robustness against highly human-like audio generated by diffusion and flow-matching TTS systems. The proposed DFADD dataset addresses this gap and provides a valuable resource for developing more resilient anti-spoofing models.
Paper Structure (20 sections, 3 figures, 3 tables)

This paper contains 20 sections, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Audio Anti-Spoofing Dataset
  4. Anti-spoofing
  5. DFADD dataset
  6. Input selection
  7. Text-to-speech model
  8. Diffusion-based Text-to-speech
  9. Flow-matching based Text-to-speech
  10. Text-to-speech synthesis
  11. Dataset comparison
  12. experimental setup
  13. Anti-spoofing model setup
  14. Evaluation setup
  15. experiments results
  16. ...and 5 more sections

Figures (3)

  • Figure 1: The pipeline of data generation for DFADD.
  • Figure 2: MOS distribution of spoofed audio generated by different TTS models (higher means more natural).
  • Figure 3: Cross-testing EER results of anti-spoofing models on DFADD test subsets. The evaluation metric is equal error rate (EER), where lower is better.