Locality enhanced dynamic biasing and sampling strategies for contextual ASR

TL;DR

This work tackles ASR performance drops on rare or context-specific phrases by improving contextual biasing (CB). It introduces locality-enhanced CB (LE-CB) with neighbourhood attention (NA) to distill bias representations and analyzes multiple sampling strategies (SMa-SMc) for CB training, using SVCCA to study representation dynamics. Across LibriSpeech and rare-word/app datasets, LE-CB variants—especially LE-CB-v2—achieve substantial relative WER reductions (approximately 25.84% on average, with larger gains on rare-word and domain datasets) while training only the CB component. The results demonstrate faster convergence, greater robustness to unseen contexts, and provide a framework for evaluating CB learning dynamics under different sampling regimes, with implications for on-device personalization of ASR systems.

Abstract

Automatic Speech Recognition (ASR) still face challenges when recognizing time-variant rare-phrases. Contextual biasing (CB) modules bias ASR model towards such contextually-relevant phrases. During training, a list of biasing phrases are selected from a large pool of phrases following a sampling strategy. In this work we firstly analyse different sampling strategies to provide insights into the training of CB for ASR with correlation plots between the bias embeddings among various training stages. Secondly, we introduce a neighbourhood attention (NA) that localizes self attention (SA) to the nearest neighbouring frames to further refine the CB output. The results show that this proposed approach provides on average a 25.84% relative WER improvement on LibriSpeech sets and rare-word evaluation compared to the baseline.

Figures (2)

• Figure 1: Locality enhanced contextual biasing
• Figure 2: Contextual bias embedding representation learning analysis with epoch-wise embedding similarity