Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Word-wise intonation model for cross-language TTS systems

Tomilov A. A., Gromova A. Y., Svischev A. N

TL;DR

A word-wise intonation model for Russian language is proposed and it is shown how it can be generalized for other languages and some practical evidence of the system robustness to parameter variations is demonstrated.

Abstract

In this paper we propose a word-wise intonation model for Russian language and show how it can be generalized for other languages. The proposed model is suitable for automatic data markup and its extended application to text-to-speech systems. It can also be implemented for an intonation contour modeling by using rule-based algorithms or by predicting contours with language models. The key idea is a partial elimination of the variability connected with different placements of a stressed syllable in a word. It is achieved with simultaneous applying of pitch simplification with a dynamic time warping clustering. The proposed model could be used as a tool for intonation research or as a backbone for prosody description in text-to-speech systems. As the advantage of the model, we show its relations with the existing intonation systems as well as the possibility of using language models for prosody prediction. Finally, we demonstrate some practical evidence of the system robustness to parameter variations.

Word-wise intonation model for cross-language TTS systems

TL;DR

A word-wise intonation model for Russian language is proposed and it is shown how it can be generalized for other languages and some practical evidence of the system robustness to parameter variations is demonstrated.

Abstract

In this paper we propose a word-wise intonation model for Russian language and show how it can be generalized for other languages. The proposed model is suitable for automatic data markup and its extended application to text-to-speech systems. It can also be implemented for an intonation contour modeling by using rule-based algorithms or by predicting contours with language models. The key idea is a partial elimination of the variability connected with different placements of a stressed syllable in a word. It is achieved with simultaneous applying of pitch simplification with a dynamic time warping clustering. The proposed model could be used as a tool for intonation research or as a backbone for prosody description in text-to-speech systems. As the advantage of the model, we show its relations with the existing intonation systems as well as the possibility of using language models for prosody prediction. Finally, we demonstrate some practical evidence of the system robustness to parameter variations.
Paper Structure (16 sections, 2 equations, 8 figures)

This paper contains 16 sections, 2 equations, 8 figures.

Table of Contents

  1. Introduction
  2. Model's description
  3. Momel splines
  4. Obtaining $F_0$ patterns
  5. The choice of a word-wise model
  6. Pitch patterns clustering
  7. Pitch processing
  8. Euclidean metric's limitations
  9. DTW clustering
  10. Intonation synthesis
  11. Connection with intonation models
  12. Language model for intonation prediction
  13. Experimental settings
  14. Results for a cluster learning for BERT models
  15. Clusters' properties with parameter variation
  16. ...and 1 more sections

Figures (8)

  • Figure 1: Overall data processing scheme
  • Figure 2: The scheme of an Intonation PAttern-STAte model. The state is an average pitch pattern position relative to the mean pitch level of a phrase. The pattern is the label of the closest cluster.
  • Figure 3: An example of $F_0$ approximation with a Momel spline and a DTW clustering. Clusters on a word can be stretched or compressed by time. Numbers indicate a cluster number.
  • Figure 4: An example of a word-wise clustering for Russian, English and Kazakh (from top to bottom). Instances for each cluster are indicated by black. Clusters' barycentres are indicated by red.
  • Figure 5: An example of INTSINT and ToRI intonation markups for a statement, a question and an exclamation in Russian
  • ...and 3 more figures