Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GALD-SE: Guided Anisotropic Lightweight Diffusion for Efficient Speech Enhancement

Chengzhong Wang, Jianjun Gu, Dingding Yao, Junfeng Li, Yonghong Yan

TL;DR

This paper proposes a method that introduces noise with anisotropic guidance during the diffusion process, allowing the neural network to preserve clean clues within noisy recordings, which substantially reduces computational complexity while exhibiting robustness against various forms of noise and speech distortion.

Abstract

Speech enhancement is designed to enhance the intelligibility and quality of speech across diverse noise conditions. Recently, diffusion model has gained lots of attention in speech enhancement area, achieving competitive results. Current diffusion-based methods blur the signal with isotropic Gaussian noise and recover clean speech from the prior. However, these methods often suffer from a substantial computational burden. We argue that the computational inefficiency partially stems from the oversight that speech enhancement is not purely a generative task; it primarily involves noise reduction and completion of missing information, while the clean clues in the original mixture do not need to be regenerated. In this paper, we propose a method that introduces noise with anisotropic guidance during the diffusion process, allowing the neural network to preserve clean clues within noisy recordings. This approach substantially reduces computational complexity while exhibiting robustness against various forms of noise and speech distortion. Experiments demonstrate that the proposed method achieves state-of-the-art results with only approximately 4.5 million parameters, a number significantly lower than that required by other diffusion methods. This effectively narrows the model size disparity between diffusion-based and predictive speech enhancement approaches. Additionally, the proposed method performs well in very noisy scenarios, demonstrating its potential for applications in highly challenging environments.

GALD-SE: Guided Anisotropic Lightweight Diffusion for Efficient Speech Enhancement

TL;DR

This paper proposes a method that introduces noise with anisotropic guidance during the diffusion process, allowing the neural network to preserve clean clues within noisy recordings, which substantially reduces computational complexity while exhibiting robustness against various forms of noise and speech distortion.

Abstract

Speech enhancement is designed to enhance the intelligibility and quality of speech across diverse noise conditions. Recently, diffusion model has gained lots of attention in speech enhancement area, achieving competitive results. Current diffusion-based methods blur the signal with isotropic Gaussian noise and recover clean speech from the prior. However, these methods often suffer from a substantial computational burden. We argue that the computational inefficiency partially stems from the oversight that speech enhancement is not purely a generative task; it primarily involves noise reduction and completion of missing information, while the clean clues in the original mixture do not need to be regenerated. In this paper, we propose a method that introduces noise with anisotropic guidance during the diffusion process, allowing the neural network to preserve clean clues within noisy recordings. This approach substantially reduces computational complexity while exhibiting robustness against various forms of noise and speech distortion. Experiments demonstrate that the proposed method achieves state-of-the-art results with only approximately 4.5 million parameters, a number significantly lower than that required by other diffusion methods. This effectively narrows the model size disparity between diffusion-based and predictive speech enhancement approaches. Additionally, the proposed method performs well in very noisy scenarios, demonstrating its potential for applications in highly challenging environments.
Paper Structure (10 sections, 9 equations, 2 figures, 2 tables)

This paper contains 10 sections, 9 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Proposed Method
  3. Stochastic Process
  4. Anisotropic Guidance
  5. Training and Sampling
  6. Experiments
  7. Datasets and Evaluation Metrics
  8. Hyperparameters and Training Settings
  9. Results
  10. Conclusion

Figures (2)

  • Figure 1: System Overview. An anisotropic guidance is derived from a coarse mask estimation network and subsequently applied to the diffusion process. The prior state is sampled using noisy speech and guidance, after which the clean output is obtained by iteratively executing the reverse process, facilitated by the diffusion network. For detailed model structures, refer to sgmse+nhs_sm_se.
  • Figure 2: Visualization of anisotropic diffusion process and its impact on speech enhancement. (a) Illustrative example of the anisotropic diffusion process. (b) Impact of anisotropic guidance on the prior state and final enhanced output.