Minimising Biasing Word Errors for Contextual ASR with the Tree-Constrained Pointer Generator

TL;DR

A novel tree-constrained pointer generator component that enables end-to-end ASR models to bias towards a list of long-tail words obtained using external contextual information, and creates a neural shortcut between the tree and the final ASR output to facilitate the recognition of the biasing words.

Abstract

Contextual knowledge is essential for reducing speech recognition errors on high-valued long-tail words. This paper proposes a novel tree-constrained pointer generator (TCPGen) component that enables end-to-end ASR models to bias towards a list of long-tail words obtained using external contextual information. With only a small overhead in memory use and computation cost, TCPGen can structure thousands of biasing words efficiently into a symbolic prefix-tree and creates a neural shortcut between the tree and the final ASR output to facilitate the recognition of the biasing words. To enhance TCPGen, we further propose a novel minimum biasing word error (MBWE) loss that directly optimises biasing word errors during training, along with a biasing-word-driven language model discounting (BLMD) method during the test. All contextual ASR systems were evaluated on the public Librispeech audiobook corpus and the data from the dialogue state tracking challenges (DSTC) with the biasing lists extracted from the dialogue-system ontology. Consistent word error rate (WER) reductions were achieved with TCPGen, which were particularly significant on the biasing words with around 40\% relative reductions in the recognition error rates. MBWE and BLMD further improved the effectiveness of TCPGen and achieved more significant WER reductions on the biasing words. TCPGen also achieved zero-shot learning of words not in the audio training set with large WER reductions on the out-of-vocabulary words in the biasing list.

Figures (5)

• Figure 1: Illustration of interpolation in TCPGen with corresponding terms in Eqn. (\ref{['eq:TCPGen_final']}). $P^\text{ptr}(y_i)$ is the TCPGen distribution. $P^\text{mdl}(y_i)$ is the distribution from a standard end-to-end model. $P(y_i)$ is the final output distribution. $\hat{P}^\text{gen}_i$ and $P^\text{gen}_i$ are the scaled and unscaled generation probabilities.
• Figure 2: An example of prefix tree search and attention in TCPGen. With previous output Tur, in_ and n are two valid wordpieces on which attention will be performed. A word end unit is denoted by _.
• Figure 3: Plots of training (left) and dev (right) set WERs across 4 training epochs. Training set WER was calculated on 5% randomly sampled utterances from the full 960-hour training set. Dev-set combines both dev-clean and dev-other sets. MBWE parameters $\mu_1, \mu_2$ were defined in Eqn. \ref{['mbr3']}.
• Figure 4: Illustration of tuning BLMD hyper-parameters for the baseline standard Conformer AED model and the Conformer AED model with TCPGen. Numbers in each grid are dev set WER in percentage. Left: Tuning $\alpha_1, \beta_1$ on the baseline model. Right: Tuning $\alpha_2, \beta_2$ on the TCPGen model with the best set of $\alpha_1, \beta_1$ found from the baseline on the left.
• Figure 5: Heat map of the generation probability for each wordpiece in an utterance taken from recognition results to show how each system spots where to use contextual biasing. Biasing words are vignette and Turner.