A DNN Based Post-Filter to Enhance the Quality of Coded Speech in MDCT Domain

TL;DR

The paper addresses quality degradation in MDCT-based low-bitrate speech coding by introducing a DNN-based post-filter that operates directly in the MDCT domain. It learns a real-valued magnitude mask derived from the MCLT spectrum using a lightweight CNN encoder-decoder and applies it to quantized MDCT coefficients before synthesis, avoiding any additional delay. The mask is defined as $M(w,k) = \frac{|X(w,k)|}{|\tilde{X}(w,k)| + \gamma}$ with the enhanced coefficient $\hat{X}_C(w,k) = M(w,k) \cdot \tilde{X}_C(w,k)$, and training relies on the MCLT magnitudes to guide learning. Evaluations at 16 kbps LC3 show an average improvement of about 10 MUSHRA points over LC3-coded speech, and gains are amplified when used with an existing LTPF, indicating complementary benefits for low-delay, low-complexity audio coding.

Abstract

Frequency domain processing, and in particular the use of Modified Discrete Cosine Transform (MDCT), is the most widespread approach to audio coding. However, at low bitrates, audio quality, especially for speech, degrades drastically due to the lack of available bits to directly code the transform coefficients. Traditionally, post-filtering has been used to mitigate artefacts in the coded speech by exploiting a-priori information of the source and extra transmitted parameters. Recently, data-driven post-filters have shown better results, but at the cost of significant additional complexity and delay. In this work, we propose a mask-based post-filter operating directly in MDCT domain of the codec, inducing no extra delay. The real-valued mask is applied to the quantized MDCT coefficients and is estimated from a relatively lightweight convolutional encoder-decoder network. Our solution is tested on the recently standardized low-delay, low-complexity codec (LC3) at lowest possible bitrate of 16 kbps. Objective and subjective assessments clearly show the advantage of this approach over the conventional post-filter, with an average improvement of 10 MUSHRA points over the LC3 coded speech.

Figures (5)

• Figure 1: System overview of the proposed DNN based post-filter
• Figure 2: POLQA score evaluation of the performance of the ideal magnitude mask on MDCT with $\alpha$ as upper limit for the mask
• Figure 3: Training and inference phase for MDCT enhancement. Fig. 3a shows the training phase where MDCT is the input to the DNN and MCLT of target and coded speech is used for loss function. Fig. 3b shows the inference phase where input and output are derived from MDCT.
• Figure 4: POLQA score evaluation of the performance of our proposed MDCT domain post-filter and its comparison to the LC3 coded speech at 16 kbps and baselines.
• Figure 5: Average MUSHRA scores (Speech) of 10 listeners with Student's-t distribution at 16 kbps.