Disentangling Pitch and Creak for Speaker Identity Preservation in Speech Synthesis

TL;DR

A system capable of faithfully modifying the perceptual voice quality of creak while preserving the speaker's perceived identity is introduced and shows greatly improved speaker verification performance over a range of creak manipulation strengths.

Abstract

We introduce a system capable of faithfully modifying the perceptual voice quality of creak while preserving the speaker's perceived identity. While it is well known that high creak probability is typically correlated with low pitch, it is important to note that this is a property observed on a population of speakers but does not necessarily hold across all situations. Disentanglement of pitch from creak is achieved by augmentation of the training dataset of a speech synthesis system with a speaker manipulation block based on conditional continuous normalizing flow. The experiments show greatly improved speaker verification performance over a range of creak manipulation strengths.

Figures (3)

• Figure 1: TTS inference with a speaker embedding manipulation block, where $\boldsymbol{\mathbf{a}}$ is the creak probability of $\boldsymbol{\mathbf{x}}$ and $\boldsymbol{\mathbf{a}} + \boldsymbol{\mathbf{\tilde{a}}}$ its modified strength
• Figure 2: Distribution of pitch for male (m) and female (f) speakers, where samples are grouped into creak (Crk) and non-creak (NCrk) categories, before and after adaptation
• Figure 3: EER as a function of the creak manipulation factor $\beta$ for the base, the adapted and adapted-2 models