Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reconstruction of the Vocal Tract from Speech via Phonetic Representations Using MRI Data

Sofiane Azzouz, Pierre-André Vuissoz, Yves Laprie

Abstract

Articulatory acoustic inversion aims to reconstruct the complete geometry of the vocal tract from the speech signal. In this paper, we present a comparative study of several levels of phonetic segmentation accuracy, together with a comparison to the baseline introduced in our previous work, which is based on Mel-Frequency Cepstral Coefficients (MFCCs). All the approaches considered are based on a denoised speech signal and aim to investigate the impact of incorporating phonetic information through three successive levels: an uncorrected automatic transcription, a temporally aligned phonetic segmentation, and an expert manual correction following alignment. The models are trained to predict articulatory contours extracted from vocal tract MRI images using an automatic contour tracking method. The results show that, among the models relying on phonetic representations, manual correction after alignment yields the best performance, approaching that of the baseline.

Reconstruction of the Vocal Tract from Speech via Phonetic Representations Using MRI Data

Abstract

Articulatory acoustic inversion aims to reconstruct the complete geometry of the vocal tract from the speech signal. In this paper, we present a comparative study of several levels of phonetic segmentation accuracy, together with a comparison to the baseline introduced in our previous work, which is based on Mel-Frequency Cepstral Coefficients (MFCCs). All the approaches considered are based on a denoised speech signal and aim to investigate the impact of incorporating phonetic information through three successive levels: an uncorrected automatic transcription, a temporally aligned phonetic segmentation, and an expert manual correction following alignment. The models are trained to predict articulatory contours extracted from vocal tract MRI images using an automatic contour tracking method. The results show that, among the models relying on phonetic representations, manual correction after alignment yields the best performance, approaching that of the baseline.
Paper Structure (13 sections, 2 equations, 3 figures, 1 table)

This paper contains 13 sections, 2 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Dataset
  3. MRI Image Preprocessing
  4. Input preprocessing
  5. Methods
  6. Model architecture
  7. Loss function
  8. Evaluation
  9. Experiments
  10. model parametres
  11. Results
  12. Discussion
  13. Conclusion

Figures (3)

  • Figure 1: Segmentation of articulators contour tracked in two images of the rt-MRI film: Arytenoid cartilage, Epiglottis, Lower lip, Vocal folds, Soft palate midline, Tongue, Upper lip, Pharyngeal wall
  • Figure 2: Model architecture
  • Figure 3: Example of two inversions compared to the original contours tracked in the rt-MRI film from baseline model. The dotted lines show the predicted contours, while the solid lines represent the original ones. The RMSE is 1.36 mm for the phoneme ’a’ on the left and 1.50 mm for ’t’ on the right.