Speech to Speech Synthesis for Voice Impersonation

TL;DR

Speech to Speech Synthesis Network (STSSN) is proposed, a model based on current state of the art systems that fuses the two disciplines in order to perform effective speech to speech style transfer for the purpose of voice impersonation.

Abstract

Numerous models have shown great success in the fields of speech recognition as well as speech synthesis, but models for speech to speech processing have not been heavily explored. We propose Speech to Speech Synthesis Network (STSSN), a model based on current state of the art systems that fuses the two disciplines in order to perform effective speech to speech style transfer for the purpose of voice impersonation. We show that our proposed model is quite powerful, and succeeds in generating realistic audio samples despite a number of drawbacks in its capacity. We benchmark our proposed model by comparing it with a generative adversarial model which accomplishes a similar task, and show that ours produces more convincing results.

Figures (6)

• Figure 1: Network output for use with CTC loss (dashed line represents best output label), taken from ctc
• Figure 2: Tacotron2 Architecture
• Figure 3: DeepSpeech Architecture
• Figure 4: Spectrograms of content (top), style (mid), and output (bottom) audio files
• Figure 5: CycleGAN Architecture