AdaCS: Adaptive Normalization for Enhanced Code-Switching ASR

TL;DR

AdaCS addresses intra-sentential code-switching in Vietnamese ASR by introducing a Bias Attention Module that couples phrase identification with normalization using a predefined bias list. The encoder and decoder are augmented with BAM to adaptively highlight and normalize CS regions, enabling robust performance in unseen domains. The work provides a new Vietnamese CS normalization dataset (50,000 training pairs, 4,000 evaluation pairs) and reports substantial WER reductions, up to $56.2\%$ on general-domain and $36.8\%$ on medical-domain CS normalization compared to prior state-of-the-art. Overall, AdaCS demonstrates strong cross-domain adaptability for low-resource CS-ASR and offers practical benefits for real-world deployment.

Abstract

Intra-sentential code-switching (CS) refers to the alternation between languages that happens within a single utterance and is a significant challenge for Automatic Speech Recognition (ASR) systems. For example, when a Vietnamese speaker uses foreign proper names or specialized terms within their speech. ASR systems often struggle to accurately transcribe intra-sentential CS due to their training on monolingual data and the unpredictable nature of CS. This issue is even more pronounced for low-resource languages, where limited data availability hinders the development of robust models. In this study, we propose AdaCS, a normalization model integrates an adaptive bias attention module (BAM) into encoder-decoder network. This novel approach provides a robust solution to CS ASR in unseen domains, thereby significantly enhancing our contribution to the field. By utilizing BAM to both identify and normalize CS phrases, AdaCS enhances its adaptive capabilities with a biased list of words provided during inference. Our method demonstrates impressive performance and the ability to handle unseen CS phrases across various domains. Experiments show that AdaCS outperforms previous state-of-the-art method on Vietnamese CS ASR normalization by considerable WER reduction of 56.2% and 36.8% on the two proposed test sets.

Figures (3)

• Figure 1: An overview of the AdaCS architecture, along with an illustrative example. The Bias Attention Module (BAM) is on the right side of the figure, and the Encode-Decode process of AdaCS is on the left side.
• Figure 2: An example of the impact of the word bias list and phrase bias list on the Tagger of AdaCS and AdapITN. Given the same input sentence, AdapITN produces consistent tagging results whether the bias list being in words (left) or phrases (right). In contrast, AdaCS adapts its tagging accordingly.
• Figure 3: Performance of AdaCS and AdapITN as the size of the bias list increases on test sets.