Direction of Arrival Correction through Speech Quality Feedback

TL;DR

This work tackles robust DOA correction for online multi-source speech enhancement by introducing a feedback loop that maximizes no-reference speech quality. It combines a location-based Demucs Denoiser with a Squim-based online quality estimator and an Adam-based optimizer to adjust the DOA feeding the beamformer. Results show real-time DOA correction is feasible for moderate localization errors, with convergence depending on the learning rate and quality-estimator stability; removing bias correction improves convergence, and initial DOA accuracy influences performance. The approach advances real-time robustness of location-informed speech enhancement and points to improvements in convergence speed, stability, and moving-source tracking as future directions.

Abstract

Real-time speech enhancement has began to rise in performance, and the Demucs Denoiser model has recently demonstrated strong performance in multiple-speech-source scenarios when accompanied by a location-based speech target selection strategy. However, it has shown to be sensitive to errors in the direction-of-arrival (DOA) estimation. In this work, a DOA correction scheme is proposed that uses the real-time estimated speech quality of its enhanced output as the observed variable in an Adam-based optimization feedback loop to find the correct DOA. In spite of the high variability of the speech quality estimation, the proposed system is able to correct in real-time an error of up to 15$^o$ using only the speech quality as its guide. Several insights are provided for future versions of the proposed system to speed up convergence and further reduce the speech quality estimation variability.

Figures (8)

• Figure 1: Overall proposed methodology.
• Figure 2: Quality estimated by Squim through time.
• Figure 3: Squim smoothened through time.
• Figure 4: Effect of $\eta$. 5 runs with $\theta_{est}=15^o$.
• Figure 5: Effect of bias correction. 30 runs with $\theta_{est}=15^o$.