Attacking Voice Anonymization Systems with Augmented Feature and Speaker Identity Difference

TL;DR

The paper tackles the vulnerability of voice anonymization by developing an attacker that can verify whether anonymized utterances come from the same speaker, despite distribution shifts. It introduces DA-SID, which fuses data from original and anonymized corpora ($D_{fused} = D_{orig} ∪ D_{anon}$) and applies SpecAugment to obtain robust embeddings $e = F(X ⊙ M_{tf})$ from ECAPA-TDNN, optimized with additive angular margin loss and, for some anonymizers, a contrastive loss; classification relies on a PLDA score $s(e_i,e_j) = \log p(e_i,e_j|H_0) - \log p(e_i,e_j|H_1)$. The contributions include a data-augmented feature representation to bridge original/anonymized distributions, a SID-focused PLDA classifier, ablations showing both components improve EER, and evaluation on LibriSpeech across six anonymizers with top-5 ranking; a workaround for mismatched T10-2 distributions uses TitaNet-Large with cosine to achieve $EER = 32.23\%$ vs $41.10\%$ baseline. The work demonstrates robust attacker performance against contemporary voice anonymization systems and informs defenses and evaluation strategies in voice privacy research.

Abstract

This study focuses on the First VoicePrivacy Attacker Challenge within the ICASSP 2025 Signal Processing Grand Challenge, which aims to develop speaker verification systems capable of determining whether two anonymized speech signals are from the same speaker. However, differences between feature distributions of original and anonymized speech complicate this task. To address this challenge, we propose an attacker system that combines Data Augmentation enhanced feature representation and Speaker Identity Difference enhanced classifier to improve verification performance, termed DA-SID. Specifically, data augmentation strategies (i.e., data fusion and SpecAugment) are utilized to mitigate feature distribution gaps, while probabilistic linear discriminant analysis (PLDA) is employed to further enhance speaker identity difference. Our system significantly outperforms the baseline, demonstrating exceptional effectiveness and robustness against various voice anonymization systems, ultimately securing a top-5 ranking in the challenge.