Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep CLAS: Deep Contextual Listen, Attend and Spell

Mengzhi Wang, Shifu Xiong, Genshun Wan, Hang Chen, Jianqing Gao, Lirong Dai

TL;DR

Deep CLAS tackles the limited use of contextual bias in Contextual-LAS by introducing a bias loss and a richer bias-attention query, enabling more precise bias utilization during decoding. It further replaces phrase-level bias encoding with fine-grained character-level representations powered by conformer-based encoding, and it explicitly fuses bias attention into the output probabilities to make contextual bias decision-making explicit. On AISHELL-1 and AISHELL-NER, Deep CLAS delivers substantial gains in recall and F1 for named-entity recognition, reporting a 53.49% relative improvement in F1 over CLAS and notable gains in recall. Overall, the approach demonstrates a practical path to robust rare-word recognition in Mandarin ASR by leveraging structured contextual bias, richer queries, and explicit probability integration.

Abstract

Contextual-LAS (CLAS) has been shown effective in improving Automatic Speech Recognition (ASR) of rare words. It relies on phrase-level contextual modeling and attention-based relevance scoring without explicit contextual constraint which lead to insufficient use of contextual information. In this work, we propose deep CLAS to use contextual information better. We introduce bias loss forcing model to focus on contextual information. The query of bias attention is also enriched to improve the accuracy of the bias attention score. To get fine-grained contextual information, we replace phrase-level encoding with character-level encoding and encode contextual information with conformer rather than LSTM. Moreover, we directly use the bias attention score to correct the output probability distribution of the model. Experiments using the public AISHELL-1 and AISHELL-NER. On AISHELL-1, compared to CLAS baselines, deep CLAS obtains a 65.78% relative recall and a 53.49% relative F1-score increase in the named entity recognition scene.

Deep CLAS: Deep Contextual Listen, Attend and Spell

TL;DR

Deep CLAS tackles the limited use of contextual bias in Contextual-LAS by introducing a bias loss and a richer bias-attention query, enabling more precise bias utilization during decoding. It further replaces phrase-level bias encoding with fine-grained character-level representations powered by conformer-based encoding, and it explicitly fuses bias attention into the output probabilities to make contextual bias decision-making explicit. On AISHELL-1 and AISHELL-NER, Deep CLAS delivers substantial gains in recall and F1 for named-entity recognition, reporting a 53.49% relative improvement in F1 over CLAS and notable gains in recall. Overall, the approach demonstrates a practical path to robust rare-word recognition in Mandarin ASR by leveraging structured contextual bias, richer queries, and explicit probability integration.

Abstract

Contextual-LAS (CLAS) has been shown effective in improving Automatic Speech Recognition (ASR) of rare words. It relies on phrase-level contextual modeling and attention-based relevance scoring without explicit contextual constraint which lead to insufficient use of contextual information. In this work, we propose deep CLAS to use contextual information better. We introduce bias loss forcing model to focus on contextual information. The query of bias attention is also enriched to improve the accuracy of the bias attention score. To get fine-grained contextual information, we replace phrase-level encoding with character-level encoding and encode contextual information with conformer rather than LSTM. Moreover, we directly use the bias attention score to correct the output probability distribution of the model. Experiments using the public AISHELL-1 and AISHELL-NER. On AISHELL-1, compared to CLAS baselines, deep CLAS obtains a 65.78% relative recall and a 53.49% relative F1-score increase in the named entity recognition scene.
Paper Structure (18 sections, 18 equations, 3 figures, 3 tables)

This paper contains 18 sections, 18 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Materials and methods
  4. Bias loss
  5. The query of bias attention
  6. The granularity of bias information
  7. Probability fusion
  8. Prefix tree
  9. EXPERIMENTS
  10. Datasets
  11. Experimental Setup
  12. Results and Discussion
  13. Bias loss
  14. The query of bias attention
  15. The granularity of bias information
  16. ...and 3 more sections

Figures (3)

  • Figure 1: The framework of CLAS model and deep CLAS model.
  • Figure 2: Prefix tree when a list of bias words consists of (“张三”,“王二”, “王小五” , “王小六”) in character level.
  • Figure 3: Bias attention map of different models, only one or two bias words were added for simplification.