Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CtrSVDD: A Benchmark Dataset and Baseline Analysis for Controlled Singing Voice Deepfake Detection

Yongyi Zang, Jiatong Shi, You Zhang, Ryuichi Yamamoto, Jionghao Han, Yuxun Tang, Shengyuan Xu, Wenxiao Zhao, Jing Guo, Tomoki Toda, Zhiyao Duan

TL;DR

CtrSVDD, a large-scale, diverse collection of bonafide and deepfake singing vocals, is introduced, and the importance of feature selection is highlighted and a need for generalization towards deepfake methods that deviate further from training distribution is highlighted.

Abstract

Recent singing voice synthesis and conversion advancements necessitate robust singing voice deepfake detection (SVDD) models. Current SVDD datasets face challenges due to limited controllability, diversity in deepfake methods, and licensing restrictions. Addressing these gaps, we introduce CtrSVDD, a large-scale, diverse collection of bonafide and deepfake singing vocals. These vocals are synthesized using state-of-the-art methods from publicly accessible singing voice datasets. CtrSVDD includes 47.64 hours of bonafide and 260.34 hours of deepfake singing vocals, spanning 14 deepfake methods and involving 164 singer identities. We also present a baseline system with flexible front-end features, evaluated against a structured train/dev/eval split. The experiments show the importance of feature selection and highlight a need for generalization towards deepfake methods that deviate further from training distribution. The CtrSVDD dataset and baselines are publicly accessible.

CtrSVDD: A Benchmark Dataset and Baseline Analysis for Controlled Singing Voice Deepfake Detection

TL;DR

CtrSVDD, a large-scale, diverse collection of bonafide and deepfake singing vocals, is introduced, and the importance of feature selection is highlighted and a need for generalization towards deepfake methods that deviate further from training distribution is highlighted.

Abstract

Recent singing voice synthesis and conversion advancements necessitate robust singing voice deepfake detection (SVDD) models. Current SVDD datasets face challenges due to limited controllability, diversity in deepfake methods, and licensing restrictions. Addressing these gaps, we introduce CtrSVDD, a large-scale, diverse collection of bonafide and deepfake singing vocals. These vocals are synthesized using state-of-the-art methods from publicly accessible singing voice datasets. CtrSVDD includes 47.64 hours of bonafide and 260.34 hours of deepfake singing vocals, spanning 14 deepfake methods and involving 164 singer identities. We also present a baseline system with flexible front-end features, evaluated against a structured train/dev/eval split. The experiments show the importance of feature selection and highlight a need for generalization towards deepfake methods that deviate further from training distribution. The CtrSVDD dataset and baselines are publicly accessible.
Paper Structure (15 sections, 3 figures, 2 tables)

This paper contains 15 sections, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. CtrSVDD dataset design
  3. Details of bonafide vocals
  4. Details of deepfake generation methods
  5. Singing voice synthesis (SVS) systems
  6. Singing voice conversion (SVC) systems
  7. Comparison with the FSD dataset
  8. Baseline systems
  9. Frontends
  10. Backend
  11. Experiments and results
  12. Experimental setup
  13. Results and discussions
  14. Conclusion
  15. Acknowledgments

Figures (3)

  • Figure 1: Histogram of audio duration. The left subfigure shows the distribution across two classes (bonafide and deepfake), whereas the right one differentiates among the train/dev/eval splits. We exclude data exceeding three standard deviations from the mean (0.6% of all data) for better visualization. All distributions are visualized with a 50% opacity then overlapped for a direct comparison between them.
  • Figure 2: Overview of source datasets and deepfake methods distribution on the train/dev/eval splits of our CtrSVDD data.
  • Figure 3: t-SNE visualization of the learned representation for the raw-waveform-based baseline system on both development and evaluation sets. Best viewed in color.