Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving ASR Contextual Biasing with Guided Attention

Jiyang Tang, Kwangyoun Kim, Suwon Shon, Felix Wu, Prashant Sridhar, Shinji Watanabe

TL;DR

This work tackles the challenge of robust contextual biasing in end-to-end ASR when bias phrase lists are large. It introduces Guided Attention (GA) as an auxiliary training loss that directly supervises cross-attention in the Contextual Adapter, implemented as GA-CE and GA-CTC variants, and combined with the Transducer loss via a weighting factor $\alpha$. Empirical results on LibriSpeech using a Conformer Transducer with Contextual Adapter show that GA losses reduce bias-related WER and preserve performance as distractor counts increase, with GA-CE typically offering stronger gains for smaller bias lists and GA-CTC providing a practical option for very large lists. Importantly, GA operates on cross-attention weights and does not add new parameters, making it straightforward to integrate with existing models. The findings demonstrate substantial relative improvements in recognition of rare vocabularies (up to $19.2\%$ over biasing baselines and $49.3\%$ over vanilla Transducers) and highlight the method’s potential to enhance production-grade contextual biasing systems.

Abstract

In this paper, we propose a Guided Attention (GA) auxiliary training loss, which improves the effectiveness and robustness of automatic speech recognition (ASR) contextual biasing without introducing additional parameters. A common challenge in previous literature is that the word error rate (WER) reduction brought by contextual biasing diminishes as the number of bias phrases increases. To address this challenge, we employ a GA loss as an additional training objective besides the Transducer loss. The proposed GA loss aims to teach the cross attention how to align bias phrases with text tokens or audio frames. Compared to studies with similar motivations, the proposed loss operates directly on the cross attention weights and is easier to implement. Through extensive experiments based on Conformer Transducer with Contextual Adapter, we demonstrate that the proposed method not only leads to a lower WER but also retains its effectiveness as the number of bias phrases increases. Specifically, the GA loss decreases the WER of rare vocabularies by up to 19.2% on LibriSpeech compared to the contextual biasing baseline, and up to 49.3% compared to a vanilla Transducer.

Improving ASR Contextual Biasing with Guided Attention

TL;DR

This work tackles the challenge of robust contextual biasing in end-to-end ASR when bias phrase lists are large. It introduces Guided Attention (GA) as an auxiliary training loss that directly supervises cross-attention in the Contextual Adapter, implemented as GA-CE and GA-CTC variants, and combined with the Transducer loss via a weighting factor . Empirical results on LibriSpeech using a Conformer Transducer with Contextual Adapter show that GA losses reduce bias-related WER and preserve performance as distractor counts increase, with GA-CE typically offering stronger gains for smaller bias lists and GA-CTC providing a practical option for very large lists. Importantly, GA operates on cross-attention weights and does not add new parameters, making it straightforward to integrate with existing models. The findings demonstrate substantial relative improvements in recognition of rare vocabularies (up to over biasing baselines and over vanilla Transducers) and highlight the method’s potential to enhance production-grade contextual biasing systems.

Abstract

In this paper, we propose a Guided Attention (GA) auxiliary training loss, which improves the effectiveness and robustness of automatic speech recognition (ASR) contextual biasing without introducing additional parameters. A common challenge in previous literature is that the word error rate (WER) reduction brought by contextual biasing diminishes as the number of bias phrases increases. To address this challenge, we employ a GA loss as an additional training objective besides the Transducer loss. The proposed GA loss aims to teach the cross attention how to align bias phrases with text tokens or audio frames. Compared to studies with similar motivations, the proposed loss operates directly on the cross attention weights and is easier to implement. Through extensive experiments based on Conformer Transducer with Contextual Adapter, we demonstrate that the proposed method not only leads to a lower WER but also retains its effectiveness as the number of bias phrases increases. Specifically, the GA loss decreases the WER of rare vocabularies by up to 19.2% on LibriSpeech compared to the contextual biasing baseline, and up to 49.3% compared to a vanilla Transducer.
Paper Structure (12 sections, 10 equations, 2 figures, 3 tables)

This paper contains 12 sections, 10 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Neural-Transducer
  4. Contextual Adapter
  5. Guided Attention Loss
  6. GA-CE Loss
  7. GA-CTC Loss
  8. Experiments
  9. Data
  10. Experimental Setup
  11. Results and Discussion
  12. Conclusion

Figures (2)

  • Figure 1: The architecture of Transducer with Contextual Adapter and the proposed Guided Attention (GA) loss. During training, gray modules are frozen, while only the yellow modules are included in back-propagation. The GA loss, highlighted in red, is newly applied to both the audio and the text biasing adapters as auxiliary losses.
  • Figure 2: B-WER of Contextual Adapter (CA) baseline, CA with GA-CE loss and CA with GA-CTC loss, using different number of distractors $N\in\{0, 100, 500, 1000, 2000\}$.