ROSE: A Recognition-Oriented Speech Enhancement Framework in Air Traffic Control Using Multi-Objective Learning
Xincheng Yu, Dongyue Guo, Jianwei Zhang, Yi Lin
TL;DR
This work tackles ATC speech echo, a domain-specific interference that degrades speech intelligibility and ASR accuracy. It introduces ROSE, a time-domain encoder–decoder U‑Net with attention-based skip fusion (ABSF) and a dual channel/sequence attention (CSAtt), trained with multi-objective losses that protect ASR-relevant features without retraining the ASR model. Key contributions are the ABSF and CSAtt modules, the ASR-oriented spectral losses, and extensive ablations demonstrating improvements in both SE and ASR across ATC data and public datasets. The approach offers a practical, plug‑and‑play enhancement for ATC systems with potential broader impact on speech interfaces in safety-critical domains, while providing interpretable attention mechanisms that reveal how features are reweighted for robust recognition.
Abstract
Radio speech echo is a specific phenomenon in the air traffic control (ATC) domain, which degrades speech quality and further impacts automatic speech recognition (ASR) accuracy. In this work, a time-domain recognition-oriented speech enhancement (ROSE) framework is proposed to improve speech intelligibility and also advance ASR accuracy based on convolutional encoder-decoder-based U-Net framework, which serves as a plug-and-play tool in ATC scenarios and does not require additional retraining of the ASR model. Specifically, 1) In the U-Net architecture, an attention-based skip-fusion (ABSF) module is applied to mine shared features from encoders using an attention mask, which enables the model to effectively fuse the hierarchical features. 2) A channel and sequence attention (CSAtt) module is innovatively designed to guide the model to focus on informative features in dual parallel attention paths, aiming to enhance the effective representations and suppress the interference noises. 3) Based on the handcrafted features, ASR-oriented optimization targets are designed to improve recognition performance in the ATC environment by learning robust feature representations. By incorporating both the SE-oriented and ASR-oriented losses, ROSE is implemented in a multi-objective learning manner by optimizing shared representations across the two task objectives. The experimental results show that the ROSE significantly outperforms other state-of-the-art methods for both the SE and ASR tasks, in which all the proposed improvements are confirmed by designed experiments. In addition, the proposed approach can contribute to the desired performance improvements on public datasets.