Towards Decoupling Frontend Enhancement and Backend Recognition in Monaural Robust ASR

TL;DR

This paper focuses on eliminating the divide between SE and ASR with an ARN (attentive recurrent network) time-domain and a CrossNet time-frequency domain enhancement models and demonstrates that ARN and CrossNet enhanced speech both translate to improved ASR results in noisy and reverberant environments, and generalize well to real acoustic scenarios.

Abstract

It has been shown that the intelligibility of noisy speech can be improved by speech enhancement (SE) algorithms. However, monaural SE has not been established as an effective frontend for automatic speech recognition (ASR) in noisy conditions compared to an ASR model trained on noisy speech directly. The divide between SE and ASR impedes the progress of robust ASR systems, especially as SE has made major advances in recent years. This paper focuses on eliminating this divide with an ARN (attentive recurrent network) time-domain and a CrossNet time-frequency domain enhancement models. The proposed systems fully decouple frontend enhancement and backend ASR trained only on clean speech. Results on the WSJ, CHiME-2, LibriSpeech, and CHiME-4 corpora demonstrate that ARN and CrossNet enhanced speech both translate to improved ASR results in noisy and reverberant environments, and generalize well to real acoustic scenarios. The proposed system outperforms the baselines trained on corrupted speech directly. Furthermore, it cuts the previous best word error rate (WER) on CHiME-2 by $28.4\%$ relatively with a $5.57\%$ WER, and achieves $3.32/4.44\%$ WER on single-channel CHiME-4 simulated/real test data without training on CHiME-4.

Figures (2)

• Figure 1: Diagram of ARN for speech enhancement. $T$ is the total number of frames and $L$ is the frame length.
• Figure 2: System architecture of a WRConformer AM. $B$ denotes the batch size, and $T$ denotes the number of time frames of the longest utterance in a batch.