Spectral Masking with Explicit Time-Context Windowing for Neural Network-Based Monaural Speech Enhancement

TL;DR

This work tackles monaural speech enhancement by enhancing spectral masking through explicit time-context windowing. It introduces a causally driven sliding-window post-processing that averages multiple context-based mask estimates, enabling improved denoising without adding network layers or training complexity. The approach is demonstrated on CRN and CDAE architectures using STFT magnitude masking, with masked reconstruction $\ ilde{S}(t,f)=\hat{M}(t,f)\odot Y(t,f)$ and a compressed magnitude loss, showing consistent gains in STOI and PESQ across diverse noise types and SNRs while adding at most ~1% parameters for convolutional models. The method is hardware-friendly and applicable as a post-processing inference step, highlighting practical improvements for real-time, resource-constrained speech enhancement systems.

Abstract

We propose and analyze the use of an explicit time-context window for neural network-based spectral masking speech enhancement to leverage signal context dependencies between neighboring frames. In particular, we concentrate on soft masking and loss computed on the time-frequency representation of the reconstructed speech. We show that the application of a time-context windowing function at both input and output of the neural network model improves the soft mask estimation process by combining multiple estimates taken from different contexts. The proposed approach is only applied as post-optimization in inference mode, not requiring additional layers or special training for the neural network model. Our results show that the method consistently increases both intelligibility and signal quality of the denoised speech, as demonstrated for two classes of convolutional-based speech enhancement models. Importantly, the proposed method requires only a negligible ($\leq1\%$) increase in the number of model parameters, making it suitable for hardware-constrained applications.

Figures (2)

• Figure 1: Sliding context window schematic example for $w=3$.
• Figure 2: Schematic of the considered convolutional recurrent neural network.