Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring VQ-VAE with Prosody Parameters for Speaker Anonymization

Sotheara Leang, Anderson Augusma, Eric Castelli, Frédérique Letué, Sethserey Sam, Dominique Vaufreydaz

TL;DR

This article investigates a novel speaker anonymization approach using an end-to-end network based on a Vector-Quantized Variational Auto-Encoder (VQ-VAE) to deal with these speech components, indicating that this method outperforms most baseline techniques in preserving emotional information.

Abstract

Human speech conveys prosody, linguistic content, and speaker identity. This article investigates a novel speaker anonymization approach using an end-to-end network based on a Vector-Quantized Variational Auto-Encoder (VQ-VAE) to deal with these speech components. This approach is designed to disentangle these components to specifically target and modify the speaker identity while preserving the linguistic and emotionalcontent. To do so, three separate branches compute embeddings for content, prosody, and speaker identity respectively. During synthesis, taking these embeddings, the decoder of the proposed architecture is conditioned on both speaker and prosody information, allowing for capturing more nuanced emotional states and precise adjustments to speaker identification. Findings indicate that this method outperforms most baseline techniques in preserving emotional information. However, it exhibits more limited performance on other voice privacy tasks, emphasizing the need for further improvements.

Exploring VQ-VAE with Prosody Parameters for Speaker Anonymization

TL;DR

This article investigates a novel speaker anonymization approach using an end-to-end network based on a Vector-Quantized Variational Auto-Encoder (VQ-VAE) to deal with these speech components, indicating that this method outperforms most baseline techniques in preserving emotional information.

Abstract

Human speech conveys prosody, linguistic content, and speaker identity. This article investigates a novel speaker anonymization approach using an end-to-end network based on a Vector-Quantized Variational Auto-Encoder (VQ-VAE) to deal with these speech components. This approach is designed to disentangle these components to specifically target and modify the speaker identity while preserving the linguistic and emotionalcontent. To do so, three separate branches compute embeddings for content, prosody, and speaker identity respectively. During synthesis, taking these embeddings, the decoder of the proposed architecture is conditioned on both speaker and prosody information, allowing for capturing more nuanced emotional states and precise adjustments to speaker identification. Findings indicate that this method outperforms most baseline techniques in preserving emotional information. However, it exhibits more limited performance on other voice privacy tasks, emphasizing the need for further improvements.
Paper Structure (13 sections, 1 figure, 5 tables)

This paper contains 13 sections, 1 figure, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Method
  4. Content Module
  5. Prosody Module
  6. Anonymization Module
  7. Decoder Module
  8. Experiments
  9. Datasets
  10. Evaluation Metrics
  11. Experimental Setup
  12. Results and Discussions
  13. Conclusion

Figures (1)

  • Figure 1: The proposed architecture: The top figure shows the encoder of the content module, while the bottom figure depicts the anonymization system, including the content, prosody, anonymization, and decoder modules. The system takes as input 80 mel-spectrogram, F0, energy, and x-vector. The pseudo-x-vector, with content and prosody embedding, is fed to the network to produce anonymized speech.