Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Thunder : Unified Regression-Diffusion Speech Enhancement with a Single Reverse Step using Brownian Bridge

Thanapat Trachu, Chawan Piansaddhayanon, Ekapol Chuangsuwanich

TL;DR

Thunder introduces a unified regression-diffusion framework for speech enhancement based on a Brownian bridge diffusion process. By predicting the clean speech $x_0$ instead of the score, the model preserves regressive capability at $t\to1$ and enables highly efficient one-step or few-step inference without increasing parameter count. The approach achieves competitive performance on VoiceBank+DEMAND, with strong generalization to out-of-domain data such as LibriFSD50k, and demonstrates reduced inference time while maintaining high speech quality. The use of interpolation between regression output and the original signal further mitigates artifacts, highlighting practical applicability for real-time enhancement scenarios.

Abstract

Diffusion-based speech enhancement has shown promising results, but can suffer from a slower inference time. Initializing the diffusion process with the enhanced audio generated by a regression-based model can be used to reduce the computational steps required. However, these approaches often necessitate a regression model, further increasing the system's complexity. We propose Thunder, a unified regression-diffusion model that utilizes the Brownian bridge process which can allow the model to act in both modes. The regression mode can be accessed by setting the diffusion time step closed to 1. However, the standard score-based diffusion modeling does not perform well in this setup due to gradient instability. To mitigate this problem, we modify the diffusion model to predict the clean speech instead of the score function, achieving competitive performance with a more compact model size and fewer reverse steps.

Thunder : Unified Regression-Diffusion Speech Enhancement with a Single Reverse Step using Brownian Bridge

TL;DR

Thunder introduces a unified regression-diffusion framework for speech enhancement based on a Brownian bridge diffusion process. By predicting the clean speech instead of the score, the model preserves regressive capability at and enables highly efficient one-step or few-step inference without increasing parameter count. The approach achieves competitive performance on VoiceBank+DEMAND, with strong generalization to out-of-domain data such as LibriFSD50k, and demonstrates reduced inference time while maintaining high speech quality. The use of interpolation between regression output and the original signal further mitigates artifacts, highlighting practical applicability for real-time enhancement scenarios.

Abstract

Diffusion-based speech enhancement has shown promising results, but can suffer from a slower inference time. Initializing the diffusion process with the enhanced audio generated by a regression-based model can be used to reduce the computational steps required. However, these approaches often necessitate a regression model, further increasing the system's complexity. We propose Thunder, a unified regression-diffusion model that utilizes the Brownian bridge process which can allow the model to act in both modes. The regression mode can be accessed by setting the diffusion time step closed to 1. However, the standard score-based diffusion modeling does not perform well in this setup due to gradient instability. To mitigate this problem, we modify the diffusion model to predict the clean speech instead of the score function, achieving competitive performance with a more compact model size and fewer reverse steps.
Paper Structure (15 sections, 11 equations, 3 figures, 4 tables)

This paper contains 15 sections, 11 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Score-based diffusion model
  3. Forward and reverse process
  4. Score-function model
  5. Inference
  6. Methodology
  7. Model parameterization
  8. Justification for the Brownian bridge process
  9. Utilizing the regression potential of the score-based model
  10. Experiments
  11. Experimental settings
  12. In-domain evaluation
  13. Out-of-domain evaluation
  14. Effect of regression mode
  15. Conclusion

Figures (3)

  • Figure 1: A summarization of Thunder during inference. The regression mode is first applied to the noisy input speech to improve the signal quality before being further refined through the diffusion mode. To reduce over-denoising artifacts caused by the regression part, the processed signal is fused with the original input to preserve its characteristic. The weights are shared across the two modes.
  • Figure 2: PESQ and SI-SDR under different numbers of diffusion time steps. Thunder performed competitively with other approaches even with just one diffusion step.
  • Figure 3: PESQ and SI-SAR of Thunder (L) at different interpolation weights $\alpha$. At high $\alpha$, performance degradation was observed due to artifacts from the regression mode, obstructing the refinement process during the diffusion mode.