Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Analysis of the Variance of Diffusion-based Speech Enhancement

Bunlong Lay, Timo Gerkmann

TL;DR

This work highlights that the scale of the variance is a dominant parameter for speech enhancement performance and shows that it controls the tradeoff between noise attenuation and speech distortions.

Abstract

Diffusion models proved to be powerful models for generative speech enhancement. In recent SGMSE+ approaches, training involves a stochastic differential equation for the diffusion process, adding both Gaussian and environmental noise to the clean speech signal gradually. The speech enhancement performance varies depending on the choice of the stochastic differential equation that controls the evolution of the mean and the variance along the diffusion processes when adding environmental and Gaussian noise. In this work, we highlight that the scale of the variance is a dominant parameter for speech enhancement performance and show that it controls the tradeoff between noise attenuation and speech distortions. More concretely, we show that a larger variance increases the noise attenuation and allows for reducing the computational footprint, as fewer function evaluations for generating the estimate are required

An Analysis of the Variance of Diffusion-based Speech Enhancement

TL;DR

This work highlights that the scale of the variance is a dominant parameter for speech enhancement performance and shows that it controls the tradeoff between noise attenuation and speech distortions.

Abstract

Diffusion models proved to be powerful models for generative speech enhancement. In recent SGMSE+ approaches, training involves a stochastic differential equation for the diffusion process, adding both Gaussian and environmental noise to the clean speech signal gradually. The speech enhancement performance varies depending on the choice of the stochastic differential equation that controls the evolution of the mean and the variance along the diffusion processes when adding environmental and Gaussian noise. In this work, we highlight that the scale of the variance is a dominant parameter for speech enhancement performance and show that it controls the tradeoff between noise attenuation and speech distortions. More concretely, we show that a larger variance increases the noise attenuation and allows for reducing the computational footprint, as fewer function evaluations for generating the estimate are required
Paper Structure (15 sections, 10 equations, 3 figures, 1 table)

This paper contains 15 sections, 10 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Diffusion models for Speech Enhancement
  3. Ornstein-Uhlenbeck with Variance Exploding (OUVE)
  4. Brownian Bridge with Exploding Diffusion Term
  5. Contribution
  6. Experimental setup
  7. Training
  8. Input representation
  9. Datasets
  10. Stochastic differential equations (SDEs)
  11. Sampling
  12. Metrics
  13. Results
  14. Conclusions
  15. Acknowledgements

Figures (3)

  • Figure 1: Mean evolutions of BBED and OUVE with parameterization as in lay202interspeech and journal respectively. Shaded areas indicate the standard deviation of BBED or OUVE.
  • Figure 2: Spectrograms of enhanced signals with BBED with different variance scales $c$ given by models from Tab. \ref{['tab:results:dns4']}. Spectrograms show the tradeoff between noise attenuation and speech component quality when increasing $c$.
  • Figure 3: Varying $t_{\text{rsp}}$ for trained BBED with fixed reverse step size $\frac{1}{30}$. Demonstrating that higher scaled variances are more robust against increasing the prior mismatch and therefore reducing computational costs.