Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DiffSSD: A Diffusion-Based Dataset For Speech Forensics

Kratika Bhagtani, Amit Kumar Singh Yadav, Paolo Bestagini, Edward J. Delp

TL;DR

The paper introduces DiffSSD, a ~200-hour, labeled diffusion-based synthetic speech dataset with 70,000 synthetic signals from 11 speakers across 10 synthesis tools (8 open-source and 2 commercial). It demonstrates that detectors trained on ASVspoof2019 fail to generalize to diffusion-based speech, and shows that retraining detectors on DiffSSD substantially improves detection across both closed-set and open-set scenarios, though some methods like LFCC-GMM remain challenging. The work highlights the practical importance of including diffusion-based and commercial generators in training datasets to ensure detector robustness against modern synthesis techniques. By making DiffSSD publicly available, the authors enable broader evaluation and development of more effective speech forensics tools in real-world settings.

Abstract

Diffusion-based speech generators are ubiquitous. These methods can generate very high quality synthetic speech and several recent incidents report their malicious use. To counter such misuse, synthetic speech detectors have been developed. Many of these detectors are trained on datasets which do not include diffusion-based synthesizers. In this paper, we demonstrate that existing detectors trained on one such dataset, ASVspoof2019, do not perform well in detecting synthetic speech from recent diffusion-based synthesizers. We propose the Diffusion-Based Synthetic Speech Dataset (DiffSSD), a dataset consisting of about 200 hours of labeled speech, including synthetic speech generated by 8 diffusion-based open-source and 2 commercial generators. We also examine the performance of existing synthetic speech detectors on DiffSSD in both closed-set and open-set scenarios. The results highlight the importance of this dataset in detecting synthetic speech generated from recent open-source and commercial speech generators.

DiffSSD: A Diffusion-Based Dataset For Speech Forensics

TL;DR

The paper introduces DiffSSD, a ~200-hour, labeled diffusion-based synthetic speech dataset with 70,000 synthetic signals from 11 speakers across 10 synthesis tools (8 open-source and 2 commercial). It demonstrates that detectors trained on ASVspoof2019 fail to generalize to diffusion-based speech, and shows that retraining detectors on DiffSSD substantially improves detection across both closed-set and open-set scenarios, though some methods like LFCC-GMM remain challenging. The work highlights the practical importance of including diffusion-based and commercial generators in training datasets to ensure detector robustness against modern synthesis techniques. By making DiffSSD publicly available, the authors enable broader evaluation and development of more effective speech forensics tools in real-world settings.

Abstract

Diffusion-based speech generators are ubiquitous. These methods can generate very high quality synthetic speech and several recent incidents report their malicious use. To counter such misuse, synthetic speech detectors have been developed. Many of these detectors are trained on datasets which do not include diffusion-based synthesizers. In this paper, we demonstrate that existing detectors trained on one such dataset, ASVspoof2019, do not perform well in detecting synthetic speech from recent diffusion-based synthesizers. We propose the Diffusion-Based Synthetic Speech Dataset (DiffSSD), a dataset consisting of about 200 hours of labeled speech, including synthetic speech generated by 8 diffusion-based open-source and 2 commercial generators. We also examine the performance of existing synthetic speech detectors on DiffSSD in both closed-set and open-set scenarios. The results highlight the importance of this dataset in detecting synthetic speech generated from recent open-source and commercial speech generators.
Paper Structure (10 sections, 2 figures, 4 tables)

This paper contains 10 sections, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. DiffSSD Dataset
  4. Generation of Text for TTS input
  5. Synthetic Speech Generation
  6. Creating Training, Validation, and Testing Sets
  7. Experiments and Results
  8. Evaluation Metric
  9. Synthetic Speech Detection
  10. Conclusion and Future Work

Figures (2)

  • Figure 1: Green and red boxes highlight some of the differences between spectrograms obtained for synthetic speech from a conventional speech generator asvdata_2019 (top) and a recent commercial generator elevenlabs2023 (bottom). Both signals have same speech content and speaker identity.
  • Figure 2: PaSST performance for each synthesizer in DiffSSD.