Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EM-TTS: Efficiently Trained Low-Resource Mongolian Lightweight Text-to-Speech

Ziqi Liang, Haoxiang Shi, Jiawei Wang, Keda Lu

TL;DR

EM-TTS tackles efficient TTS for a low-resource language (Mongolian) by deploying a fully CNN-based, two-stage pipeline that first maps text to a coarse mel spectrogram (Text2Spectrum) and then refines it to a complete spectrum (SSRN). It introduces guided attention and a trio of data augmentations (DN, SA, SR) to boost robustness with limited data. On the NCMMSC2022-MTTSC Mongolian dataset, EM-TTS achieves competitive naturalness and intelligibility while substantially reducing training time and parameter count compared with autoregressive and transformer baselines, demonstrating practical benefits for on-device deployment. Overall, the approach offers a concrete path to fast, lightweight TTS for low-resource languages, enabling broader accessibility and deployment options.”

Abstract

Recently, deep learning-based Text-to-Speech (TTS) systems have achieved high-quality speech synthesis results. Recurrent neural networks have become a standard modeling technique for sequential data in TTS systems and are widely used. However, training a TTS model which includes RNN components requires powerful GPU performance and takes a long time. In contrast, CNN-based sequence synthesis techniques can significantly reduce the parameters and training time of a TTS model while guaranteeing a certain performance due to their high parallelism, which alleviate these economic costs of training. In this paper, we propose a lightweight TTS system based on deep convolutional neural networks, which is a two-stage training end-to-end TTS model and does not employ any recurrent units. Our model consists of two stages: Text2Spectrum and SSRN. The former is used to encode phonemes into a coarse mel spectrogram and the latter is used to synthesize the complete spectrum from the coarse mel spectrogram. Meanwhile, we improve the robustness of our model by a series of data augmentations, such as noise suppression, time warping, frequency masking and time masking, for solving the low resource mongolian problem. Experiments show that our model can reduce the training time and parameters while ensuring the quality and naturalness of the synthesized speech compared to using mainstream TTS models. Our method uses NCMMSC2022-MTTSC Challenge dataset for validation, which significantly reduces training time while maintaining a certain accuracy.

EM-TTS: Efficiently Trained Low-Resource Mongolian Lightweight Text-to-Speech

TL;DR

EM-TTS tackles efficient TTS for a low-resource language (Mongolian) by deploying a fully CNN-based, two-stage pipeline that first maps text to a coarse mel spectrogram (Text2Spectrum) and then refines it to a complete spectrum (SSRN). It introduces guided attention and a trio of data augmentations (DN, SA, SR) to boost robustness with limited data. On the NCMMSC2022-MTTSC Mongolian dataset, EM-TTS achieves competitive naturalness and intelligibility while substantially reducing training time and parameter count compared with autoregressive and transformer baselines, demonstrating practical benefits for on-device deployment. Overall, the approach offers a concrete path to fast, lightweight TTS for low-resource languages, enabling broader accessibility and deployment options.”

Abstract

Recently, deep learning-based Text-to-Speech (TTS) systems have achieved high-quality speech synthesis results. Recurrent neural networks have become a standard modeling technique for sequential data in TTS systems and are widely used. However, training a TTS model which includes RNN components requires powerful GPU performance and takes a long time. In contrast, CNN-based sequence synthesis techniques can significantly reduce the parameters and training time of a TTS model while guaranteeing a certain performance due to their high parallelism, which alleviate these economic costs of training. In this paper, we propose a lightweight TTS system based on deep convolutional neural networks, which is a two-stage training end-to-end TTS model and does not employ any recurrent units. Our model consists of two stages: Text2Spectrum and SSRN. The former is used to encode phonemes into a coarse mel spectrogram and the latter is used to synthesize the complete spectrum from the coarse mel spectrogram. Meanwhile, we improve the robustness of our model by a series of data augmentations, such as noise suppression, time warping, frequency masking and time masking, for solving the low resource mongolian problem. Experiments show that our model can reduce the training time and parameters while ensuring the quality and naturalness of the synthesized speech compared to using mainstream TTS models. Our method uses NCMMSC2022-MTTSC Challenge dataset for validation, which significantly reduces training time while maintaining a certain accuracy.
Paper Structure (17 sections, 6 equations, 6 figures, 4 tables)

This paper contains 17 sections, 6 equations, 6 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Text2Spectrum Module
  5. Spectrogram Super-Resolution Module
  6. Guided Attention
  7. Data Augmentation
  8. Noise Suppression (DN)
  9. SpecArgument (SA)
  10. Spectrogram-Resize (SR)
  11. Experiments and Results
  12. Experimental Setup
  13. Results
  14. Ablation Study
  15. Discussion
  16. ...and 2 more sections

Figures (6)

  • Figure 1: The architecture of our proposed EM-TTS
  • Figure 2: Noise Suppression using DCCRN model
  • Figure 3: Mel Spectrogram by SpecArgument
  • Figure 4: Resize ratio < 1
  • Figure 5: Resize ratio > 1
  • ...and 1 more figures