Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Two-Stage Framework in Cross-Spectrum Domain for Real-Time Speech Enhancement

Yuewei Zhang, Huanbin Zou, Jie Zhu

TL;DR

The paper addresses real-time speech enhancement by introducing FDFNet, a two-stage cross-spectrum framework that first enhances the STFT magnitude and then refines the spectrum in the STDCT domain. The STDCT-based second stage leverages a real-valued spectrum to implicitly recover phase and provides stronger residual noise suppression via TFSM-enhanced DSR-Net, achieving state-of-the-art performance among causal two-stage methods. Experimental results on VoiceBank+DEMAND show superior WB-PESQ, CBAK, and COVL metrics compared with prior systems, while maintaining real-time capability. This cross-domain fusion approach offers practical gains for low-latency SE applications and reduces the computational burden of explicit phase estimation.

Abstract

Two-stage pipeline is popular in speech enhancement tasks due to its superiority over traditional single-stage methods. The current two-stage approaches usually enhance the magnitude spectrum in the first stage, and further modify the complex spectrum to suppress the residual noise and recover the speech phase in the second stage. The above whole process is performed in the short-time Fourier transform (STFT) spectrum domain. In this paper, we re-implement the above second sub-process in the short-time discrete cosine transform (STDCT) spectrum domain. The reason is that we have found STDCT performs greater noise suppression capability than STFT. Additionally, the implicit phase of STDCT ensures simpler and more efficient phase recovery, which is challenging and computationally expensive in the STFT-based methods. Therefore, we propose a novel two-stage framework called the STFT-STDCT spectrum fusion network (FDFNet) for speech enhancement in cross-spectrum domain. Experimental results demonstrate that the proposed FDFNet outperforms the previous two-stage methods and also exhibits superior performance compared to other advanced systems.

A Two-Stage Framework in Cross-Spectrum Domain for Real-Time Speech Enhancement

TL;DR

The paper addresses real-time speech enhancement by introducing FDFNet, a two-stage cross-spectrum framework that first enhances the STFT magnitude and then refines the spectrum in the STDCT domain. The STDCT-based second stage leverages a real-valued spectrum to implicitly recover phase and provides stronger residual noise suppression via TFSM-enhanced DSR-Net, achieving state-of-the-art performance among causal two-stage methods. Experimental results on VoiceBank+DEMAND show superior WB-PESQ, CBAK, and COVL metrics compared with prior systems, while maintaining real-time capability. This cross-domain fusion approach offers practical gains for low-latency SE applications and reduces the computational burden of explicit phase estimation.

Abstract

Two-stage pipeline is popular in speech enhancement tasks due to its superiority over traditional single-stage methods. The current two-stage approaches usually enhance the magnitude spectrum in the first stage, and further modify the complex spectrum to suppress the residual noise and recover the speech phase in the second stage. The above whole process is performed in the short-time Fourier transform (STFT) spectrum domain. In this paper, we re-implement the above second sub-process in the short-time discrete cosine transform (STDCT) spectrum domain. The reason is that we have found STDCT performs greater noise suppression capability than STFT. Additionally, the implicit phase of STDCT ensures simpler and more efficient phase recovery, which is challenging and computationally expensive in the STFT-based methods. Therefore, we propose a novel two-stage framework called the STFT-STDCT spectrum fusion network (FDFNet) for speech enhancement in cross-spectrum domain. Experimental results demonstrate that the proposed FDFNet outperforms the previous two-stage methods and also exhibits superior performance compared to other advanced systems.
Paper Structure (11 sections, 11 equations, 2 figures, 2 tables)

This paper contains 11 sections, 11 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Proposed Method
  3. The overall architecture
  4. STDCT-based spectrum refinement sub-network (DSR-Net)
  5. Loss function
  6. Experiments and Results
  7. Experimental setting
  8. Ablation study
  9. Comparison with previous advanced systems
  10. Conclusions
  11. Acknowledge

Figures (2)

  • Figure 1: Overall structure of the STFT-STDCT spectrum fusion network (FDFNet).
  • Figure 2: (a) The details of STDCT-based spectrum refinement sub-network (DSR-Net). (b) The details of time-frequency sequence modeling (TFSM) block.