Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Diffusion-based Speech Enhancement with Schrödinger Bridge and Symmetric Noise Schedule

Siyi Wang, Siyi Liu, Andrew Harper, Paul Kendrick, Mathieu Salzmann, Milos Cernak

TL;DR

This work introduces Schrödinger Bridge-based Speech Enhancement (SBSE), a diffusion-based method that learns the transformation directly from noisy speech to clean speech, preserving structure absent in Gaussian-prior methods. It combines a ratio mask prediction stage with a Schrödinger Bridge diffusion stage and employs a symmetric noise schedule to achieve strong performance, particularly in very low SNR conditions, with only a few inference steps (as few as five). The approach is validated on the DNS 2023 dataset, where SBSE and its ratio-mask extension outperform both discriminative baselines and prior diffusion methods, and a notable advantage is the improved quality at low SNRs along with competitive speed relative to other diffusion models. The method relies on a DDPM-style sampler guided by SB dynamics and demonstrates the practical potential of simulation-free SB learning for speech enhancement, albeit with some artifacts under extreme noise that warrant future work.

Abstract

Recently, diffusion-based generative models have demonstrated remarkable performance in speech enhancement tasks. However, these methods still encounter challenges, including the lack of structural information and poor performance in low Signal-to-Noise Ratio (SNR) scenarios. To overcome these challenges, we propose the Schröodinger Bridge-based Speech Enhancement (SBSE) method, which learns the diffusion processes directly between the noisy input and the clean distribution, unlike conventional diffusion-based speech enhancement systems that learn data to Gaussian distributions. To enhance performance in extremely noisy conditions, we introduce a two-stage system incorporating ratio mask information into the diffusion-based generative model. Our experimental results show that our proposed SBSE method outperforms all the baseline models and achieves state-of-the-art performance, especially in low SNR conditions. Importantly, only a few inference steps are required to achieve the best result.

Diffusion-based Speech Enhancement with Schrödinger Bridge and Symmetric Noise Schedule

TL;DR

This work introduces Schrödinger Bridge-based Speech Enhancement (SBSE), a diffusion-based method that learns the transformation directly from noisy speech to clean speech, preserving structure absent in Gaussian-prior methods. It combines a ratio mask prediction stage with a Schrödinger Bridge diffusion stage and employs a symmetric noise schedule to achieve strong performance, particularly in very low SNR conditions, with only a few inference steps (as few as five). The approach is validated on the DNS 2023 dataset, where SBSE and its ratio-mask extension outperform both discriminative baselines and prior diffusion methods, and a notable advantage is the improved quality at low SNRs along with competitive speed relative to other diffusion models. The method relies on a DDPM-style sampler guided by SB dynamics and demonstrates the practical potential of simulation-free SB learning for speech enhancement, albeit with some artifacts under extreme noise that warrant future work.

Abstract

Recently, diffusion-based generative models have demonstrated remarkable performance in speech enhancement tasks. However, these methods still encounter challenges, including the lack of structural information and poor performance in low Signal-to-Noise Ratio (SNR) scenarios. To overcome these challenges, we propose the Schröodinger Bridge-based Speech Enhancement (SBSE) method, which learns the diffusion processes directly between the noisy input and the clean distribution, unlike conventional diffusion-based speech enhancement systems that learn data to Gaussian distributions. To enhance performance in extremely noisy conditions, we introduce a two-stage system incorporating ratio mask information into the diffusion-based generative model. Our experimental results show that our proposed SBSE method outperforms all the baseline models and achieves state-of-the-art performance, especially in low SNR conditions. Importantly, only a few inference steps are required to achieve the best result.
Paper Structure (17 sections, 9 equations, 2 figures, 1 table)

This paper contains 17 sections, 9 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Score-based Generative Models
  4. Schrödinger Bridge Model
  5. Schrödinger Bridge Problem
  6. Simulation-free Framework
  7. Method
  8. Ratio Mask Prediction Model
  9. Schrödinger Bridge Model
  10. Training Task
  11. Inference Procedure
  12. Experiments
  13. Experimental Setup
  14. Baselines
  15. Speech Quality Assessment
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Architecture of the proposed two-stage method. The Schrödinger Bridge (SB) can take a predicted mask as an auxiliary input.
  • Figure 2: MUSHRA subjective evaluation with 19 participants.