Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SNAP: Speaker Nulling for Artifact Projection in Speech Deepfake Detection

Kyudan Jung, Jihwan Kim, Minwoo Lee, Soyoon Kim, Jeonghoon Kim, Jaegul Choo, Cheonbok Park

Abstract

Recent advancements in text-to-speech technologies enable generating high-fidelity synthetic speech nearly indistinguishable from real human voices. While recent studies show the efficacy of self-supervised learning-based speech encoders for deepfake detection, these models struggle to generalize across unseen speakers. Our quantitative analysis suggests these encoder representations are substantially influenced by speaker information, causing detectors to exploit speaker-specific correlations rather than artifact-related cues. We call this phenomenon speaker entanglement. To mitigate this reliance, we introduce SNAP, a speaker-nulling framework. We estimate a speaker subspace and apply orthogonal projection to suppress speaker-dependent components, isolating synthesis artifacts within the residual features. By reducing speaker entanglement, SNAP encourages detectors to focus on artifact-related patterns, leading to state-of-the-art performance.

SNAP: Speaker Nulling for Artifact Projection in Speech Deepfake Detection

Abstract

Recent advancements in text-to-speech technologies enable generating high-fidelity synthetic speech nearly indistinguishable from real human voices. While recent studies show the efficacy of self-supervised learning-based speech encoders for deepfake detection, these models struggle to generalize across unseen speakers. Our quantitative analysis suggests these encoder representations are substantially influenced by speaker information, causing detectors to exploit speaker-specific correlations rather than artifact-related cues. We call this phenomenon speaker entanglement. To mitigate this reliance, we introduce SNAP, a speaker-nulling framework. We estimate a speaker subspace and apply orthogonal projection to suppress speaker-dependent components, isolating synthesis artifacts within the residual features. By reducing speaker entanglement, SNAP encourages detectors to focus on artifact-related patterns, leading to state-of-the-art performance.
Paper Structure (17 sections, 11 equations, 3 figures, 2 tables)

This paper contains 17 sections, 11 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Speaker Entanglement Phenomenon
  3. Discriminative Capacity of WavLM Representations for Speaker Identity
  4. Subspace Decomposition
  5. Methodology
  6. Feature Extraction and Normalization
  7. Speaker Subspace Nulling
  8. Classification
  9. Inference
  10. Experiments and Results
  11. Experimental Setup
  12. Main Results
  13. Quantification of Speaker Entanglement
  14. Robustness in Unseen TTS Scenarios
  15. Impact of Nulling Speaker Information
  16. ...and 2 more sections

Figures (3)

  • Figure 1: t-SNE visualization of WavLM-Large feature embeddings for real and synthetic speech samples. (a) Each color represents a distinct speaker identity. (b) The same embedding space as in (a), colored by audio type (real vs. synthetic).
  • Figure 2: Silhouette score comparison of baseline and SNAP features clustered by (a) speaker identity, (b) natural versus synthetic speech. Shaded areas represent silhouette coefficients of each data sample and dashed lines denote mean scores of all samples. The mean speaker clustering score drops from 0.026 to -0.002. The natural versus synthetic clustering mean increases from 0.118 to 0.181. SNAP effectively suppresses speaker identity, isolating synthesis artifacts.
  • Figure 3: Comparison of Deepfake speech detection error rate between the Baseline (WavLM-Large) and SNAP as the number of training speakers increases.