Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Text Injection for Neural Contextual Biasing

Zhong Meng, Zelin Wu, Rohit Prabhavalkar, Cal Peyser, Weiran Wang, Nanxin Chen, Tara N. Sainath, Bhuvana Ramabhadran

TL;DR

This work tackles the challenge of recognizing contextually relevant but rare phrases in end-to-end ASR by introducing contextual text injection (CTI), which converts large unpaired text into speech-like representations and uses them to train a neural biasing component. CTI enables joint optimization of the ASR model and its biasing mechanism with abundant text data, addressing data sparsity and scalability of prior approaches. A contextual text-injected MWER (CTI-MWER) training objective further reduces errors caused by contextual biasing by generating N-best hypotheses from both audio-text pairs and unpaired text. Across large-scale experiments with up to 100B unpaired sentences, CTI delivers substantial WER reductions (up to 43.3% relative in-context) and CTI-MWER adds a further ~23.5% improvement, demonstrating strong, scalable gains for contextual ASR in real-world domains.

Abstract

Neural contextual biasing effectively improves automatic speech recognition (ASR) for crucial phrases within a speaker's context, particularly those that are infrequent in the training data. This work proposes contextual text injection (CTI) to enhance contextual ASR. CTI leverages not only the paired speech-text data, but also a much larger corpus of unpaired text to optimize the ASR model and its biasing component. Unpaired text is converted into speech-like representations and used to guide the model's attention towards relevant bias phrases. Moreover, we introduce a contextual text-injected (CTI) minimum word error rate (MWER) training, which minimizes the expected WER caused by contextual biasing when unpaired text is injected into the model. Experiments show that CTI with 100 billion text sentences can achieve up to 43.3% relative WER reduction from a strong neural biasing model. CTI-MWER provides a further relative improvement of 23.5%.

Text Injection for Neural Contextual Biasing

TL;DR

This work tackles the challenge of recognizing contextually relevant but rare phrases in end-to-end ASR by introducing contextual text injection (CTI), which converts large unpaired text into speech-like representations and uses them to train a neural biasing component. CTI enables joint optimization of the ASR model and its biasing mechanism with abundant text data, addressing data sparsity and scalability of prior approaches. A contextual text-injected MWER (CTI-MWER) training objective further reduces errors caused by contextual biasing by generating N-best hypotheses from both audio-text pairs and unpaired text. Across large-scale experiments with up to 100B unpaired sentences, CTI delivers substantial WER reductions (up to 43.3% relative in-context) and CTI-MWER adds a further ~23.5% improvement, demonstrating strong, scalable gains for contextual ASR in real-world domains.

Abstract

Neural contextual biasing effectively improves automatic speech recognition (ASR) for crucial phrases within a speaker's context, particularly those that are infrequent in the training data. This work proposes contextual text injection (CTI) to enhance contextual ASR. CTI leverages not only the paired speech-text data, but also a much larger corpus of unpaired text to optimize the ASR model and its biasing component. Unpaired text is converted into speech-like representations and used to guide the model's attention towards relevant bias phrases. Moreover, we introduce a contextual text-injected (CTI) minimum word error rate (MWER) training, which minimizes the expected WER caused by contextual biasing when unpaired text is injected into the model. Experiments show that CTI with 100 billion text sentences can achieve up to 43.3% relative WER reduction from a strong neural biasing model. CTI-MWER provides a further relative improvement of 23.5%.
Paper Structure (13 sections, 9 equations, 1 figure, 3 tables)

This paper contains 13 sections, 9 equations, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Text Injection
  4. Neural Contextual Biasing with Deferred NAM
  5. Contextual Text Injection
  6. Contextual Text-Injected MWER
  7. Experiments
  8. Data
  9. Modeling Architecture
  10. Text injection for Neural Contextual Biasing
  11. MWER Training for Neural Contextual Biasing
  12. Scalability to Number of Bias Phrases
  13. Conclusion

Figures (1)

  • Figure 1: Contextual text injection (CTI) for a speech transducer with neural biasing. Blue and red arrows show the forward propagation paths for paired audio-transcript data and unpaired text, respectively.