Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DIN-CTS: Low-Complexity Depthwise-Inception Neural Network with Contrastive Training Strategy for Deepfake Speech Detection

Lam Pham, Dat Tran, Phat Lam, Florian Skopik, Alexander Schindler, Silvia Poletti, David Fischinger, Martin Boyer

TL;DR

This work tackles deepfake speech detection with a low-complexity DIN that uses depthwise and inception-inspired blocks to efficiently learn discriminative features. A three-stage Contrastive Training Strategy (CTS) combines multi-class, contrastive, and bonafide-variance objectives, culminating in a Gaussian model over bonafide embeddings for inference. On ASVspoof 2019 LA, the single-model DIN with CTS (M3) achieves an EER of $4.6\%$, accuracy of $95.4\%$, F1 of $97.3\%$, and AUC of $98.9\%$ with only $1.77$ million parameters and $985$ MFLOPS, outperforming top single-system submissions. The approach uses a Mahalanobis distance to compare test embeddings with the bonafide distribution, enabling potential real-time deployment for DSD tasks. Key contributions include the low-complexity DIN architecture, the three-stage CTS, and the probabilistic inference framework for robust detection with unseen generators, supported by detailed ablations and visualizations.

Abstract

In this paper, we propose a deep neural network approach for deepfake speech detection (DSD) based on a lowcomplexity Depthwise-Inception Network (DIN) trained with a contrastive training strategy (CTS). In this framework, input audio recordings are first transformed into spectrograms using Short-Time Fourier Transform (STFT) and Linear Filter (LF), which are then used to train the DIN. Once trained, the DIN processes bonafide utterances to extract audio embeddings, which are used to construct a Gaussian distribution representing genuine speech. Deepfake detection is then performed by computing the distance between a test utterance and this distribution to determine whether the utterance is fake or bonafide. To evaluate our proposed systems, we conducted extensive experiments on the benchmark dataset of ASVspoof 2019 LA. The experimental results demonstrate the effectiveness of combining the Depthwise-Inception Network with the contrastive learning strategy in distinguishing between fake and bonafide utterances. We achieved Equal Error Rate (EER), Accuracy (Acc.), F1, AUC scores of 4.6%, 95.4%, 97.3%, and 98.9% respectively using a single, low-complexity DIN with just 1.77 M parameters and 985 M FLOPS on short audio segments (4 seconds). Furthermore, our proposed system outperforms the single-system submissions in the ASVspoof 2019 LA challenge, showcasing its potential for real-time applications.

DIN-CTS: Low-Complexity Depthwise-Inception Neural Network with Contrastive Training Strategy for Deepfake Speech Detection

TL;DR

This work tackles deepfake speech detection with a low-complexity DIN that uses depthwise and inception-inspired blocks to efficiently learn discriminative features. A three-stage Contrastive Training Strategy (CTS) combines multi-class, contrastive, and bonafide-variance objectives, culminating in a Gaussian model over bonafide embeddings for inference. On ASVspoof 2019 LA, the single-model DIN with CTS (M3) achieves an EER of , accuracy of , F1 of , and AUC of with only million parameters and MFLOPS, outperforming top single-system submissions. The approach uses a Mahalanobis distance to compare test embeddings with the bonafide distribution, enabling potential real-time deployment for DSD tasks. Key contributions include the low-complexity DIN architecture, the three-stage CTS, and the probabilistic inference framework for robust detection with unseen generators, supported by detailed ablations and visualizations.

Abstract

In this paper, we propose a deep neural network approach for deepfake speech detection (DSD) based on a lowcomplexity Depthwise-Inception Network (DIN) trained with a contrastive training strategy (CTS). In this framework, input audio recordings are first transformed into spectrograms using Short-Time Fourier Transform (STFT) and Linear Filter (LF), which are then used to train the DIN. Once trained, the DIN processes bonafide utterances to extract audio embeddings, which are used to construct a Gaussian distribution representing genuine speech. Deepfake detection is then performed by computing the distance between a test utterance and this distribution to determine whether the utterance is fake or bonafide. To evaluate our proposed systems, we conducted extensive experiments on the benchmark dataset of ASVspoof 2019 LA. The experimental results demonstrate the effectiveness of combining the Depthwise-Inception Network with the contrastive learning strategy in distinguishing between fake and bonafide utterances. We achieved Equal Error Rate (EER), Accuracy (Acc.), F1, AUC scores of 4.6%, 95.4%, 97.3%, and 98.9% respectively using a single, low-complexity DIN with just 1.77 M parameters and 985 M FLOPS on short audio segments (4 seconds). Furthermore, our proposed system outperforms the single-system submissions in the ASVspoof 2019 LA challenge, showcasing its potential for real-time applications.

Paper Structure

This paper contains 11 sections, 5 equations, 4 figures, 2 tables.

Table of Contents

  1. INTRODUCTION
  2. Depthwise-Inception Network Architecture and Contrastive Training Strategy
  3. Spectrogram Feature Extraction
  4. Depthwise-Inception Network Architecture
  5. Contrastive Training Strategy
  6. Inference Process
  7. Experiments and Results
  8. Datasets and evaluation metrics
  9. Experimental settings
  10. Results and discussion
  11. Conclusion

Figures (4)

  • Figure 1: The proposed Depthwise-Inception Network architecture and Contrastive Training Strategy
  • Figure 2: Histogram of Mahalanobis distances between the training bonafide distribution ('Train' subset) and the test bonafide & fake utterances ('Evaluating' subset) with the model M1 on the left side and the model M3 on the right side
  • Figure 3: TSNE-based visualization of the output feature maps extracted from the backbone (embedding $\boldsymbol{X}$ extracted from model M1 on the left side and extracted from model M3 on the right side) among testing bonafide and fake utterances ('Evaluation' subset with A07 to A19 generators)
  • Figure :