Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Anonymizing Speech: Evaluating and Designing Speaker Anonymization Techniques

Pierre Champion

TL;DR

This work tackles the privacy risks posed by speech data by formalizing and evaluating speaker anonymization. It advances the field by analyzing how target speaker choices influence privacy assessment, proposing feature-level disentanglement (notably through vector quantization) to better separate speaker identity from linguistic content, and introducing new attack models to test invertibility. The study systematically compares signal-processing, voice-conversion, and adversarial approaches under the VoicePrivacy framework, highlighting that target selection, data, and evaluation protocols critically shape measured privacy. It also proposes novel metrics, including invertibility attacks and mispronunciation-based utility measures, to provide a more faithful picture of privacy-utility trade-offs in real-world deployments. Collectively, the findings underscore that robust privacy requires careful evaluation design, advanced disentanglement techniques such as VQ-based methods, and consideration of multi-faceted attack models to prevent identity leakage while preserving intelligible speech.

Abstract

The growing use of voice user interfaces has led to a surge in the collection and storage of speech data. While data collection allows for the development of efficient tools powering most speech services, it also poses serious privacy issues for users as centralized storage makes private personal speech data vulnerable to cyber threats. With the increasing use of voice-based digital assistants like Amazon's Alexa, Google's Home, and Apple's Siri, and with the increasing ease with which personal speech data can be collected, the risk of malicious use of voice-cloning and speaker/gender/pathological/etc. recognition has increased. This thesis proposes solutions for anonymizing speech and evaluating the degree of the anonymization. In this work, anonymization refers to making personal speech data unlinkable to an identity while maintaining the usefulness (utility) of the speech signal (e.g., access to linguistic content). We start by identifying several challenges that evaluation protocols need to consider to evaluate the degree of privacy protection properly. We clarify how anonymization systems must be configured for evaluation purposes and highlight that many practical deployment configurations do not permit privacy evaluation. Furthermore, we study and examine the most common voice conversion-based anonymization system and identify its weak points before suggesting new methods to overcome some limitations. We isolate all components of the anonymization system to evaluate the degree of speaker PPI associated with each of them. Then, we propose several transformation methods for each component to reduce as much as possible speaker PPI while maintaining utility. We promote anonymization algorithms based on quantization-based transformation as an alternative to the most-used and well-known noise-based approach. Finally, we endeavor a new attack method to invert anonymization.

Anonymizing Speech: Evaluating and Designing Speaker Anonymization Techniques

TL;DR

This work tackles the privacy risks posed by speech data by formalizing and evaluating speaker anonymization. It advances the field by analyzing how target speaker choices influence privacy assessment, proposing feature-level disentanglement (notably through vector quantization) to better separate speaker identity from linguistic content, and introducing new attack models to test invertibility. The study systematically compares signal-processing, voice-conversion, and adversarial approaches under the VoicePrivacy framework, highlighting that target selection, data, and evaluation protocols critically shape measured privacy. It also proposes novel metrics, including invertibility attacks and mispronunciation-based utility measures, to provide a more faithful picture of privacy-utility trade-offs in real-world deployments. Collectively, the findings underscore that robust privacy requires careful evaluation design, advanced disentanglement techniques such as VQ-based methods, and consideration of multi-faceted attack models to prevent identity leakage while preserving intelligible speech.

Abstract

The growing use of voice user interfaces has led to a surge in the collection and storage of speech data. While data collection allows for the development of efficient tools powering most speech services, it also poses serious privacy issues for users as centralized storage makes private personal speech data vulnerable to cyber threats. With the increasing use of voice-based digital assistants like Amazon's Alexa, Google's Home, and Apple's Siri, and with the increasing ease with which personal speech data can be collected, the risk of malicious use of voice-cloning and speaker/gender/pathological/etc. recognition has increased. This thesis proposes solutions for anonymizing speech and evaluating the degree of the anonymization. In this work, anonymization refers to making personal speech data unlinkable to an identity while maintaining the usefulness (utility) of the speech signal (e.g., access to linguistic content). We start by identifying several challenges that evaluation protocols need to consider to evaluate the degree of privacy protection properly. We clarify how anonymization systems must be configured for evaluation purposes and highlight that many practical deployment configurations do not permit privacy evaluation. Furthermore, we study and examine the most common voice conversion-based anonymization system and identify its weak points before suggesting new methods to overcome some limitations. We isolate all components of the anonymization system to evaluate the degree of speaker PPI associated with each of them. Then, we propose several transformation methods for each component to reduce as much as possible speaker PPI while maintaining utility. We promote anonymization algorithms based on quantization-based transformation as an alternative to the most-used and well-known noise-based approach. Finally, we endeavor a new attack method to invert anonymization.
Paper Structure (161 sections, 31 equations, 50 figures, 15 tables)

This paper contains 161 sections, 31 equations, 50 figures, 15 tables.

Table of Contents

  1. State of the art
  2. Speech and neural networks
  3. Introduction
  4. Speech production
  5. Speech production mechanism
  6. The different components and aspects of speech
  7. Speech processing
  8. Time-frequency processing
  9. Short-term features
  10. Long-term features
  11. Artificial neural networks
  12. Bottleneck layer
  13. Residual network
  14. Time delayed neural and convolutional neural networks
  15. Factorized version
  16. ...and 146 more sections

Figures (50)

  • Figure 1: The production mechanisms of voiced sounds as a source-filter model.
  • Figure 2: The process of extracting a speech frame spectrum.
  • Figure 3: The process of computing mfcc.
  • Figure 4: Feedforward multilayer perceptron.
  • Figure 5: Computations for a tdnn in dotted and solid lines, and sub-sampled tdnn in solid lines only. Frames $t - 2$ through $t + 2$ are spliced together at the input layer (which can be written as context {-2, -1, 0, 1, 2} or more compactly as [-2, 2]); and then at the upper three hidden layers with splice offsets of {-1, 1}, {-2, 0} and {-1, 1}.
  • ...and 45 more figures