Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GTR-Voice: Articulatory Phonetics Informed Controllable Expressive Speech Synthesis

Zehua Kcriss Li, Meiying Melissa Chen, Yi Zhong, Pinxin Liu, Zhiyao Duan

TL;DR

This work defines a framework with three dimensions: Glottalization, Tenseness, and Resonance (GTR), to guide the synthesis at the voice production level, and demonstrates precise controllability along the GTR dimensions on two fine-tuned expressive TTS models.

Abstract

Expressive speech synthesis aims to generate speech that captures a wide range of para-linguistic features, including emotion and articulation, though current research primarily emphasizes emotional aspects over the nuanced articulatory features mastered by professional voice actors. Inspired by this, we explore expressive speech synthesis through the lens of articulatory phonetics. Specifically, we define a framework with three dimensions: Glottalization, Tenseness, and Resonance (GTR), to guide the synthesis at the voice production level. With this framework, we record a high-quality speech dataset named GTR-Voice, featuring 20 Chinese sentences articulated by a professional voice actor across 125 distinct GTR combinations. We verify the framework and GTR annotations through automatic classification and listening tests, and demonstrate precise controllability along the GTR dimensions on two fine-tuned expressive TTS models. We open-source the dataset and TTS models.

GTR-Voice: Articulatory Phonetics Informed Controllable Expressive Speech Synthesis

TL;DR

This work defines a framework with three dimensions: Glottalization, Tenseness, and Resonance (GTR), to guide the synthesis at the voice production level, and demonstrates precise controllability along the GTR dimensions on two fine-tuned expressive TTS models.

Abstract

Expressive speech synthesis aims to generate speech that captures a wide range of para-linguistic features, including emotion and articulation, though current research primarily emphasizes emotional aspects over the nuanced articulatory features mastered by professional voice actors. Inspired by this, we explore expressive speech synthesis through the lens of articulatory phonetics. Specifically, we define a framework with three dimensions: Glottalization, Tenseness, and Resonance (GTR), to guide the synthesis at the voice production level. With this framework, we record a high-quality speech dataset named GTR-Voice, featuring 20 Chinese sentences articulated by a professional voice actor across 125 distinct GTR combinations. We verify the framework and GTR annotations through automatic classification and listening tests, and demonstrate precise controllability along the GTR dimensions on two fine-tuned expressive TTS models. We open-source the dataset and TTS models.
Paper Structure (13 sections, 1 figure, 1 table)

This paper contains 13 sections, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. The GTR-Voice dataset
  3. Articulatory phonetics inspired dimensions
  4. Recording process
  5. Label Verification
  6. GTR controllable speech synthesis
  7. FastPitch
  8. StyleTTS
  9. Experiments
  10. Evaluation Setup
  11. Results and Discussions
  12. Conclusions and future work
  13. Acknowledgements

Figures (1)

  • Figure 1: Average classification accuracy across subjects and utterances of the subjective evaluation of GTR controllability for FastPitch and StyleTTS. Sub-figures from left to right show results of GTR variations along different dimensions around the value of the reference utterance. We do not include the Whisper voices (G=0 & R=0) as only Tenseness can vary, making it unfair to compare with other GTR variations, based on the evaluation process we conduct.