Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Temporal Variability and Multi-Viewed Self-Supervised Representations to Tackle the ASVspoof5 Deepfake Challenge

Yuankun Xie, Xiaopeng Wang, Zhiyong Wang, Ruibo Fu, Zhengqi Wen, Haonan Cheng, Long Ye

TL;DR

The paper addresses open-condition audio deepfake detection in ASVspoof5 Track1, proposing a multi-faceted countermeasure pipeline that combines data expansion, data augmentation (notably Freqmask), SSL feature extraction, and multi-view fusion. By systematically evaluating datasets, augmentation strategies, SSL features, and duration choices, the authors demonstrate significant improvements on the evaluation progress set, achieving $minDCF$=$0.0158$ and $EER$=$0.55\%$ through temporal and SSL feature fusion. However, a notable drop on the full evaluation set reveals generalization challenges to unseen deepfake and codec techniques, underscoring the need for further work to close the gap between progress-set and full-set performance. Overall, the work highlights the value of leveraging temporal diversity and multiple SSL perspectives to enhance open-condition audio deepfake detection, with practical implications for robust audio security systems.

Abstract

ASVspoof5, the fifth edition of the ASVspoof series, is one of the largest global audio security challenges. It aims to advance the development of countermeasure (CM) to discriminate bonafide and spoofed speech utterances. In this paper, we focus on addressing the problem of open-domain audio deepfake detection, which corresponds directly to the ASVspoof5 Track1 open condition. At first, we comprehensively investigate various CM on ASVspoof5, including data expansion, data augmentation, and self-supervised learning (SSL) features. Due to the high-frequency gaps characteristic of the ASVspoof5 dataset, we introduce Frequency Mask, a data augmentation method that masks specific frequency bands to improve CM robustness. Combining various scale of temporal information with multiple SSL features, our experiments achieved a minDCF of 0.0158 and an EER of 0.55% on the ASVspoof 5 Track 1 evaluation progress set.

Temporal Variability and Multi-Viewed Self-Supervised Representations to Tackle the ASVspoof5 Deepfake Challenge

TL;DR

The paper addresses open-condition audio deepfake detection in ASVspoof5 Track1, proposing a multi-faceted countermeasure pipeline that combines data expansion, data augmentation (notably Freqmask), SSL feature extraction, and multi-view fusion. By systematically evaluating datasets, augmentation strategies, SSL features, and duration choices, the authors demonstrate significant improvements on the evaluation progress set, achieving = and = through temporal and SSL feature fusion. However, a notable drop on the full evaluation set reveals generalization challenges to unseen deepfake and codec techniques, underscoring the need for further work to close the gap between progress-set and full-set performance. Overall, the work highlights the value of leveraging temporal diversity and multiple SSL perspectives to enhance open-condition audio deepfake detection, with practical implications for robust audio security systems.

Abstract

ASVspoof5, the fifth edition of the ASVspoof series, is one of the largest global audio security challenges. It aims to advance the development of countermeasure (CM) to discriminate bonafide and spoofed speech utterances. In this paper, we focus on addressing the problem of open-domain audio deepfake detection, which corresponds directly to the ASVspoof5 Track1 open condition. At first, we comprehensively investigate various CM on ASVspoof5, including data expansion, data augmentation, and self-supervised learning (SSL) features. Due to the high-frequency gaps characteristic of the ASVspoof5 dataset, we introduce Frequency Mask, a data augmentation method that masks specific frequency bands to improve CM robustness. Combining various scale of temporal information with multiple SSL features, our experiments achieved a minDCF of 0.0158 and an EER of 0.55% on the ASVspoof 5 Track 1 evaluation progress set.
Paper Structure (21 sections, 4 figures, 8 tables, 1 algorithm)

This paper contains 21 sections, 4 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Countermeasure
  3. Data expansion
  4. Data augmentation
  5. Freqmask data augmentation method
  6. Other data augmentation method
  7. Self-Supervised learning feature
  8. SSL feature selection
  9. Reconsider the time duration length
  10. Backbone
  11. Experiments
  12. Details of ASVspoof5 dataset
  13. Implementation details
  14. Results and Discussion
  15. Initial results for data expansion and feature selection
  16. ...and 6 more sections

Figures (4)

  • Figure 1: The original spectrogram of ASVspoof5 evaluation set shows high-frequency band gaps. (a) and (b) exhibit gaps above 4kHz, while (c) and (d) exhibit gaps above 7kHz.
  • Figure 2: Comparison between the DA techniques using Freqmask and a low-pass filter. (a) shows the original spectrogram, (b) and (c) show the spectrograms with 4 kHz and 7 kHz Freqmask applied, respectively, and (d) shows the spectrogram with a 4 kHz low-pass filter applied.
  • Figure 3: The duration distribution statistics of the ASVspoof5 dataset shows the audio duration on the horizontal axis and the number of occurrences in the dataset (frequency) on the vertical axis.
  • Figure 4: The duration distribution statistics of the ASVspoof5 dataset.