Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards the Synthesis of Non-speech Vocalizations

Enjamamul Hoq, Ifeoma Nwogu

TL;DR

This work tackles the unconditional synthesis of non-speech vocalizations, focusing on infant cries, by applying DiffWave, a diffusion probabilistic model trained on two infant-cry datasets. The model learns a reverse denoising process to generate audio from Gaussian noise, optimizing an ELBO that reduces to predicting the denoising noise, and utilizes 200 diffusion steps with diffusion-step conditioning via a sinusoidal embedding. Key contributions include dataset-driven analysis for infant cries, a detailed non-autoregressive architecture with bidirectional dilated convolutions, and practical fast-sampling techniques that reduce reverse steps to as few as 6 while maintaining quality. The results demonstrate high-fidelity, diverse cry generation at 16 kHz, offering a data-augmentation path for pediatric acoustic research and potential privacy-preserving synthetic data generation.

Abstract

In this report, we focus on the unconditional generation of infant cry sounds using the DiffWave framework, which has shown great promise in generating high-quality audio from noise. We use two distinct datasets of infant cries: the Baby Chillanto and the deBarbaro cry dataset. These datasets are used to train the DiffWave model to generate new cry sounds that maintain high fidelity and diversity. The focus here is on DiffWave's capability to handle the unconditional generation task.

Towards the Synthesis of Non-speech Vocalizations

TL;DR

This work tackles the unconditional synthesis of non-speech vocalizations, focusing on infant cries, by applying DiffWave, a diffusion probabilistic model trained on two infant-cry datasets. The model learns a reverse denoising process to generate audio from Gaussian noise, optimizing an ELBO that reduces to predicting the denoising noise, and utilizes 200 diffusion steps with diffusion-step conditioning via a sinusoidal embedding. Key contributions include dataset-driven analysis for infant cries, a detailed non-autoregressive architecture with bidirectional dilated convolutions, and practical fast-sampling techniques that reduce reverse steps to as few as 6 while maintaining quality. The results demonstrate high-fidelity, diverse cry generation at 16 kHz, offering a data-augmentation path for pediatric acoustic research and potential privacy-preserving synthetic data generation.

Abstract

In this report, we focus on the unconditional generation of infant cry sounds using the DiffWave framework, which has shown great promise in generating high-quality audio from noise. We use two distinct datasets of infant cries: the Baby Chillanto and the deBarbaro cry dataset. These datasets are used to train the DiffWave model to generate new cry sounds that maintain high fidelity and diversity. The focus here is on DiffWave's capability to handle the unconditional generation task.

Paper Structure

This paper contains 13 sections, 11 equations, 3 figures, 2 algorithms.

Table of Contents

  1. Dataset
  2. Methodology
  3. Forward Process
  4. Reverse Process
  5. The Loss Function
  6. Model Architecture
  7. Dilated Convolutions
  8. Residual Layers
  9. Conditioning on Diffusion Steps
  10. Training Setup
  11. Training Process
  12. Sampling Process
  13. Results

Figures (3)

  • Figure 1: The diffusion and reverse process in diffusion probabilistic models. Figure taken from kong2020diffwave.
  • Figure 2: The network architecture of DiffWave in modeling $\epsilon_\theta$. Figure taken from kong2020diffwave.
  • Figure 3: Generated samples from model trained on a) deBarbaro dataset b) Baby Chillanto dataset.