Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EDSep: An Effective Diffusion-Based Method for Speech Source Separation

Jinwei Dong, Xinsheng Wang, Qirong Mao

TL;DR

EDSep addresses slow convergence and suboptimal quality in diffusion-based single-channel speech separation by adopting a variance-exploding SDE, a noise-conditioned denoiser, and a stochastic sampler. The method uses a permutation-aware training strategy and a time-domain diffusion process with a non-linear STFT transform to improve separation quality and efficiency. Empirical results on WSJ0-2mix, LRS2-2mix, and VoxCeleb2-2mix show that EDSep outperforms DiffSep and is competitive with Conv-TasNet on several metrics, demonstrating the practicality of diffusion-based approaches for real-world speech separation.

Abstract

Generative models have attracted considerable attention for speech separation tasks, and among these, diffusion-based methods are being explored. Despite the notable success of diffusion techniques in generation tasks, their adaptation to speech separation has encountered challenges, notably slow convergence and suboptimal separation outcomes. To address these issues and enhance the efficacy of diffusion-based speech separation, we introduce EDSep, a novel single-channel method grounded in score matching via stochastic differential equation (SDE). This method enhances generative modeling for speech source separation by optimizing training and sampling efficiency. Specifically, a novel denoiser function is proposed to approximate data distributions, which obtains ideal denoiser outputs. Additionally, a stochastic sampler is carefully designed to resolve the reverse SDE during the sampling process, gradually separating speech from mixtures. Extensive experiments on databases such as WSJ0-2mix, LRS2-2mix, and VoxCeleb2-2mix demonstrate our proposed method's superior performance over existing diffusion and discriminative models, validating its efficacy.

EDSep: An Effective Diffusion-Based Method for Speech Source Separation

TL;DR

EDSep addresses slow convergence and suboptimal quality in diffusion-based single-channel speech separation by adopting a variance-exploding SDE, a noise-conditioned denoiser, and a stochastic sampler. The method uses a permutation-aware training strategy and a time-domain diffusion process with a non-linear STFT transform to improve separation quality and efficiency. Empirical results on WSJ0-2mix, LRS2-2mix, and VoxCeleb2-2mix show that EDSep outperforms DiffSep and is competitive with Conv-TasNet on several metrics, demonstrating the practicality of diffusion-based approaches for real-world speech separation.

Abstract

Generative models have attracted considerable attention for speech separation tasks, and among these, diffusion-based methods are being explored. Despite the notable success of diffusion techniques in generation tasks, their adaptation to speech separation has encountered challenges, notably slow convergence and suboptimal separation outcomes. To address these issues and enhance the efficacy of diffusion-based speech separation, we introduce EDSep, a novel single-channel method grounded in score matching via stochastic differential equation (SDE). This method enhances generative modeling for speech source separation by optimizing training and sampling efficiency. Specifically, a novel denoiser function is proposed to approximate data distributions, which obtains ideal denoiser outputs. Additionally, a stochastic sampler is carefully designed to resolve the reverse SDE during the sampling process, gradually separating speech from mixtures. Extensive experiments on databases such as WSJ0-2mix, LRS2-2mix, and VoxCeleb2-2mix demonstrate our proposed method's superior performance over existing diffusion and discriminative models, validating its efficacy.

Paper Structure

This paper contains 13 sections, 21 equations, 2 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. SDE for Diffusion-Based Modelling
  4. Denoising Score Matching
  5. Related Work in Diffusion-Based Speech Separation
  6. Method
  7. Training Procedure
  8. Sampling Procedure
  9. Implementation Details
  10. Experiments
  11. Datasets
  12. Results
  13. Conclusion

Figures (2)

  • Figure 1: The pipeline of diffusion models for speech separation. In training process, speech sources $\bm{x}_0$ gradually add Gaussian noise and we train the model to approximate the score function. In sampling process, we sample $\bm{x}_{T}$ from the mixture distributions and we use the score-based model to separate mixed speech $\bm{y}$ step by step.
  • Figure 2: Spectrograms of the clean sources and the separated speech of Conv-TasNet, DiffSep and our method. The sample was chosen randomly from the LRS2-2mix dataset.