Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A comparative study of generative models for child voice conversion

Protima Nomo Sudro, Anton Ragni, Thomas Hain

TL;DR

This work compares four generative approaches—GAN-based CycleGAN-VC2, VAE, flow-based, and diffusion models—for adult-to-child voice conversion using acted dubbing data. It introduces a frequency-warping post-processing step to reduce mismatch with target child speech and evaluates both objective metrics (MCD, F0 RMSE) and subjective MOS. The results show CycleGAN-VC2 and diffusion models deliver strong objective performance, with warping consistently improving similarity across models; diffusion benefits most from larger training data. The findings highlight the trade-offs between one-to-one mappings and multi-speaker capabilities for dubbing applications, guiding model selection based on data availability and target deployment.

Abstract

Generative models are a popular choice for adult-to-adult voice conversion (VC) because of their efficient way of modelling unlabelled data. To this point their usefulness in producing children speech and in particular adult to child VC has not been investigated. For adult to child VC, four generative models are compared: diffusion model, flow based model, variational autoencoders, and generative adversarial network. Results show that although converted speech outputs produce by those models appear plausible, they exhibit insufficient similarity with the target speaker characteristics. We introduce an efficient frequency warping technique that can be applied to the output of models, and which shows significant reduction of the mismatch between adult and child. The output of all the models are evaluated using both objective and subjective measures. In particular we compare specific speaker pairing using a unique corpus collected for dubbing of children speech.

A comparative study of generative models for child voice conversion

TL;DR

This work compares four generative approaches—GAN-based CycleGAN-VC2, VAE, flow-based, and diffusion models—for adult-to-child voice conversion using acted dubbing data. It introduces a frequency-warping post-processing step to reduce mismatch with target child speech and evaluates both objective metrics (MCD, F0 RMSE) and subjective MOS. The results show CycleGAN-VC2 and diffusion models deliver strong objective performance, with warping consistently improving similarity across models; diffusion benefits most from larger training data. The findings highlight the trade-offs between one-to-one mappings and multi-speaker capabilities for dubbing applications, guiding model selection based on data availability and target deployment.

Abstract

Generative models are a popular choice for adult-to-adult voice conversion (VC) because of their efficient way of modelling unlabelled data. To this point their usefulness in producing children speech and in particular adult to child VC has not been investigated. For adult to child VC, four generative models are compared: diffusion model, flow based model, variational autoencoders, and generative adversarial network. Results show that although converted speech outputs produce by those models appear plausible, they exhibit insufficient similarity with the target speaker characteristics. We introduce an efficient frequency warping technique that can be applied to the output of models, and which shows significant reduction of the mismatch between adult and child. The output of all the models are evaluated using both objective and subjective measures. In particular we compare specific speaker pairing using a unique corpus collected for dubbing of children speech.

Paper Structure

This paper contains 14 sections, 4 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Investigation of voice conversion models
  3. Generative adversarial network
  4. Variational autoencoder
  5. Flow based generative model
  6. Diffusion model
  7. Post-processing
  8. Experiments
  9. Corpus
  10. Implementation details
  11. Evaluation metrics
  12. Results and discussion
  13. Subjective evaluation
  14. Conclusion

Figures (5)

  • Figure 1: Inference stage of the CycleGAN based VC system
  • Figure 2: Inference stage of the VAE based VC system
  • Figure 3: Flow based VC system
  • Figure 4: Diffusion model based VC system
  • Figure 5: Average MCD values for different amount of training data (1 hours and 5 hours) used