Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving the Speaker Anonymization Evaluation's Robustness to Target Speakers with Adversarial Learning

Carlos Franzreb, Arnab Das, Tim Polzehl, Sebastian Möller

TL;DR

The paper addresses overestimated privacy in speaker anonymization evaluations that arise when same-gender target selection is used, due to leaked target information in anonymized speech. It introduces a target classifier trained alongside the source recognizer and employs a gradient reversal layer to adversarially suppress target information, improving the reliability of privacy assessments. Experiments across two anonymizers and TSAs show that target information is disproportionately encoded and that the proposed adversarial approach yields more consistent privacy estimates, especially under same-gender TSA; results generalize across evaluation setups. This approach provides a practical, low-overhead improvement to privacy evaluation, enabling more trustworthy assessments and better-informed deployment decisions for anonymization systems.

Abstract

The current privacy evaluation for speaker anonymization often overestimates privacy when a same-gender target selection algorithm (TSA) is used, although this TSA leaks the speaker's gender and should hence be more vulnerable. We hypothesize that this occurs because the evaluation does not account for the fact that anonymized speech contains information from both the source and target speakers. To address this, we propose to add a target classifier that measures the influence of target speaker information in the evaluation, which can also be removed with adversarial learning. Experiments demonstrate that this approach is effective for multiple anonymizers, particularly when using a same-gender TSA, leading to a more reliable assessment.

Improving the Speaker Anonymization Evaluation's Robustness to Target Speakers with Adversarial Learning

TL;DR

The paper addresses overestimated privacy in speaker anonymization evaluations that arise when same-gender target selection is used, due to leaked target information in anonymized speech. It introduces a target classifier trained alongside the source recognizer and employs a gradient reversal layer to adversarially suppress target information, improving the reliability of privacy assessments. Experiments across two anonymizers and TSAs show that target information is disproportionately encoded and that the proposed adversarial approach yields more consistent privacy estimates, especially under same-gender TSA; results generalize across evaluation setups. This approach provides a practical, low-overhead improvement to privacy evaluation, enabling more trustworthy assessments and better-informed deployment decisions for anonymization systems.

Abstract

The current privacy evaluation for speaker anonymization often overestimates privacy when a same-gender target selection algorithm (TSA) is used, although this TSA leaks the speaker's gender and should hence be more vulnerable. We hypothesize that this occurs because the evaluation does not account for the fact that anonymized speech contains information from both the source and target speakers. To address this, we propose to add a target classifier that measures the influence of target speaker information in the evaluation, which can also be removed with adversarial learning. Experiments demonstrate that this approach is effective for multiple anonymizers, particularly when using a same-gender TSA, leading to a more reliable assessment.

Paper Structure

This paper contains 10 sections, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Privacy evaluation
  3. Proposed architecture
  4. Experiment setup
  5. Private kNN-VC
  6. Experiments
  7. Using a larger recognizer
  8. Source and target information in the baseline
  9. Informing the recognizer about targets
  10. Conclusion

Figures (4)

  • Figure 1: Diagram of the privacy evaluation, including the proposed target classifier, whose gradients can be back-propagated to the recognizer through a gradient reversal layer (GRL).
  • Figure 2: Different Private kNN-VC configurations evaluated with SpAnE and VPC24. The latter overestimates privacy for the same-gender TSA.
  • Figure 3: Classifier VERs per epoch for the baseline, for the two TSAs. The recognizer learns more about targets than about sources.
  • Figure 4: Final source VERs depending on for how many epochs $w_T$ is zero. The lowest VER is achieved for $E=6$, for both TSAs.