Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Investigating the Impact of Speech Enhancement on Audio Deepfake Detection in Noisy Environments

Anacin, Angela, Shruti Kshirsagar, Anderson R. Avila

Abstract

Logical Access (LA) attacks, also known as audio deepfake attacks, use Text-to-Speech (TTS) or Voice Conversion (VC) methods to generate spoofed speech data. This can represent a serious threat to Automatic Speaker Verification (ASV) systems, as intruders can use such attacks to bypass voice biometric security. In this study, we investigate the correlation between speech quality and the performance of audio spoofing detection systems (i.e., LA task). For that, the performance of two enhancement algorithms is evaluated based on two perceptual speech quality measures, namely Perceptual Evaluation of Speech Quality (PESQ) and Speech-to-Reverberation Modulation Ratio (SRMR), and in respect to their impact on the audio spoofing detection system. We adopted the LA dataset, provided in the ASVspoof 2019 Challenge, and corrupted its test set with different Signal-to-Noise Ratio (SNR) levels, while leaving the training data untouched. Enhancement was applied to attenuate the detrimental effects of noisy speech, and the performances of two models, Speech Enhancement Generative Adversarial Network (SEGAN) and Metric-Optimized Generative Adversarial Network Plus (MetricGAN+), were compared. Although we expect that speech quality will correlate well with speech applications' performance, it can also have as a side effect on downstream tasks if unwanted artifacts are introduced or relevant information is removed from the speech signal. Our results corroborate with this hypothesis, as we found that the enhancement algorithm leading to the highest speech quality scores, MetricGAN+, provided the lowest Equal Error Rate (EER) on the audio spoofing detection task, whereas the enhancement method with the lowest speech quality scores, SEGAN, led to the lowest EER, thus leading to better performance on the LA task.

Investigating the Impact of Speech Enhancement on Audio Deepfake Detection in Noisy Environments

Abstract

Logical Access (LA) attacks, also known as audio deepfake attacks, use Text-to-Speech (TTS) or Voice Conversion (VC) methods to generate spoofed speech data. This can represent a serious threat to Automatic Speaker Verification (ASV) systems, as intruders can use such attacks to bypass voice biometric security. In this study, we investigate the correlation between speech quality and the performance of audio spoofing detection systems (i.e., LA task). For that, the performance of two enhancement algorithms is evaluated based on two perceptual speech quality measures, namely Perceptual Evaluation of Speech Quality (PESQ) and Speech-to-Reverberation Modulation Ratio (SRMR), and in respect to their impact on the audio spoofing detection system. We adopted the LA dataset, provided in the ASVspoof 2019 Challenge, and corrupted its test set with different Signal-to-Noise Ratio (SNR) levels, while leaving the training data untouched. Enhancement was applied to attenuate the detrimental effects of noisy speech, and the performances of two models, Speech Enhancement Generative Adversarial Network (SEGAN) and Metric-Optimized Generative Adversarial Network Plus (MetricGAN+), were compared. Although we expect that speech quality will correlate well with speech applications' performance, it can also have as a side effect on downstream tasks if unwanted artifacts are introduced or relevant information is removed from the speech signal. Our results corroborate with this hypothesis, as we found that the enhancement algorithm leading to the highest speech quality scores, MetricGAN+, provided the lowest Equal Error Rate (EER) on the audio spoofing detection task, whereas the enhancement method with the lowest speech quality scores, SEGAN, led to the lowest EER, thus leading to better performance on the LA task.
Paper Structure (12 sections, 4 figures, 1 table)

This paper contains 12 sections, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methods and Materials
  3. Speech Enhancement Methods
  4. Instrumental Quality Measures
  5. Spoofed Speech Detection Model
  6. Experimental Details
  7. Corpus Description
  8. Benchmark and Evaluation Setup
  9. Corrupted Speech and Enhancement
  10. Figures-of-merit
  11. Experimental Results
  12. Conclusion

Figures (4)

  • Figure 1: Training and Testing Setup; x(n) refers to the raw audio, h(n) refers to convolutive noise being added, Gs and Gt refer to Graph Spectral and Graph Temporal, these combine and pass into the Heterogeneous Stacking Graph Attention Layer (HS-GAL).
  • Figure 2: SNR for real and spoof audio signal at 0dB
  • Figure 3: Perceptual quality performance of two enhancement algorithms based on PESQ and SRMR.
  • Figure 4: Correlation between PESQ/SRMR scores and EER, t-DCF for SEGAN and MetricGAN+. (a–d) show correlations between PESQ vs. SEGAN/MetricGAN+ for EER and t-DCF, (e–h) show correlations between SRMR vs. SEGAN/MetricGAN+ for EER and t-DCF.