Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

InterBiasing: Boost Unseen Word Recognition through Biasing Intermediate Predictions

Yu Nakagome, Michael Hentschel

TL;DR

This paper proposes an adaptation parameter-free approach based on Self-conditioned CTC that improves the recognition accuracy of misrecognized target keywords by substituting their intermediate CTC predictions with corrected labels, which are then passed on to the subsequent layers.

Abstract

Despite recent advances in end-to-end speech recognition methods, their output is biased to the training data's vocabulary, resulting in inaccurate recognition of unknown terms or proper nouns. To improve the recognition accuracy for a given set of such terms, we propose an adaptation parameter-free approach based on Self-conditioned CTC. Our method improves the recognition accuracy of misrecognized target keywords by substituting their intermediate CTC predictions with corrected labels, which are then passed on to the subsequent layers. First, we create pairs of correct labels and recognition error instances for a keyword list using Text-to-Speech and a recognition model. We use these pairs to replace intermediate prediction errors by the labels. Conditioning the subsequent layers of the encoder on the labels, it is possible to acoustically evaluate the target keywords. Experiments conducted in Japanese demonstrated that our method successfully improved the F1 score for unknown words.

InterBiasing: Boost Unseen Word Recognition through Biasing Intermediate Predictions

TL;DR

This paper proposes an adaptation parameter-free approach based on Self-conditioned CTC that improves the recognition accuracy of misrecognized target keywords by substituting their intermediate CTC predictions with corrected labels, which are then passed on to the subsequent layers.

Abstract

Despite recent advances in end-to-end speech recognition methods, their output is biased to the training data's vocabulary, resulting in inaccurate recognition of unknown terms or proper nouns. To improve the recognition accuracy for a given set of such terms, we propose an adaptation parameter-free approach based on Self-conditioned CTC. Our method improves the recognition accuracy of misrecognized target keywords by substituting their intermediate CTC predictions with corrected labels, which are then passed on to the subsequent layers. First, we create pairs of correct labels and recognition error instances for a keyword list using Text-to-Speech and a recognition model. We use these pairs to replace intermediate prediction errors by the labels. Conditioning the subsequent layers of the encoder on the labels, it is possible to acoustically evaluate the target keywords. Experiments conducted in Japanese demonstrated that our method successfully improved the F1 score for unknown words.
Paper Structure (14 sections, 13 equations, 2 figures, 2 tables)

This paper contains 14 sections, 13 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Connectionist Temporal Classification
  4. Conformer Encoder
  5. Self-conditioned CTC
  6. InterBiasing: Biasing Intermediate Predictions
  7. Collecting Recognition Errors for a Keyword List with Synthetic Audio
  8. Biasing Intermediate Predictions
  9. Experiments
  10. Data
  11. Model Configurations
  12. Results
  13. Analysis of the Relationship Between Beam Size and Keyword Recognition Rate
  14. Conclusions

Figures (2)

  • Figure 1: Overview of the proposed InterBiasing methods. The proposed method has two steps. Step 1) Speech for a keyword list is generated via Text-to-Speech (TTS), and this speech is fed into a recognition model to create pairs of correct labels and recognition error instances. Step 2) These pairs are utilized to replace intermediate prediction errors with the correct labels, and the subsequent layers infer recognition hypothesis based on these labels.
  • Figure 2: F1 scores of Out-of-Vocabulary (OOV) and Non-OOV words in JSUT basic 5000 with different beam sizes. LM shallow fusion and Keyword-boosted Beam Search (KBBS) were utilized. Beam size was set to 2, 3, 5, 10, 20, respectively.