Parallel Tacotron: Non-Autoregressive and Controllable TTS

TL;DR

The paper tackles the efficiency gap between autoregressive TTS and modern parallelizable models by introducing Parallel Tacotron, a non-autoregressive TTS with a variational residual encoder to capture prosody. It combines global speaker-specific and phoneme-level VAEs, lightweight convolutions, and an iterative spectrogram loss to preserve naturalness while enabling efficient parallel synthesis. Experiments show Parallel Tacotron achieves MOS and perceptual preferences on par with Tacotron 2, while delivering roughly 13× faster inference on TPUs, with VAEs and iterative loss providing additional gains. The approach demonstrates that non-autoregressive TTS can rival autoregressive baselines in naturalness and significantly improve deployment practicality on real-time systems.

Abstract

Although neural end-to-end text-to-speech models can synthesize highly natural speech, there is still room for improvements to its efficiency and naturalness. This paper proposes a non-autoregressive neural text-to-speech model augmented with a variational autoencoder-based residual encoder. This model, called \emph{Parallel Tacotron}, is highly parallelizable during both training and inference, allowing efficient synthesis on modern parallel hardware. The use of the variational autoencoder relaxes the one-to-many mapping nature of the text-to-speech problem and improves naturalness. To further improve the naturalness, we use lightweight convolutions, which can efficiently capture local contexts, and introduce an iterative spectrogram loss inspired by iterative refinement. Experimental results show that Parallel Tacotron matches a strong autoregressive baseline in subjective evaluations with significantly decreased inference time.

Figures (1)

• Figure 1: Block diagram of the Parallel Tacotron model. The residual encoder (purple blocks) in this figure correspond to the global VAE in Section \ref{['sec:global_vae']}.