Neural Codec-based Adversarial Sample Detection for Speaker Verification

TL;DR

ASV systems are vulnerable to adversarial inputs; the paper introduces a defense that uses neural codecs to purify audio before ASV scoring. The method computes a score difference $d = |s - s'|$ between the ASV score $s$ on the original utterance and $s'$ after codec re-synthesis, and uses a threshold $\tau_{det}$ set to a user-specified false-positive rate to detect adversarial samples. Across 15 neural codec variants, the Descript-audio-codec (DAC) achieves the highest detection rate and outperforms recent SOTA detectors with a single model, while incurring minimal degradation on genuine samples. This attack-agnostic detector offers a practical, scalable defense for security-critical ASV deployments.

Abstract

Automatic Speaker Verification (ASV), increasingly used in security-critical applications, faces vulnerabilities from rising adversarial attacks, with few effective defenses available. In this paper, we propose a neural codec-based adversarial sample detection method for ASV. The approach leverages the codec's ability to discard redundant perturbations and retain essential information. Specifically, we distinguish between genuine and adversarial samples by comparing ASV score differences between original and re-synthesized audio (by codec models). This comprehensive study explores all open-source neural codecs and their variant models for experiments. The Descript-audio-codec model stands out by delivering the highest detection rate among 15 neural codecs and surpassing seven prior state-of-the-art (SOTA) detection methods. Note that, our single-model method even outperforms a SOTA ensemble method by a large margin.

Figures (3)

• Figure 1: The neural codec detection framework. $s$ and $s'$ are the ASV scores for $x$ and $x'$. The absolute value $|s - s'|$ between $s$ and $s'$ is for detection.
• Figure 2: The score-difference distributions.
• Figure 3: The trade-off for neural codec purification.