Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CIF-T: A Novel CIF-based Transducer Architecture for Automatic Speech Recognition

Tian-Hao Zhang, Dinghao Zhou, Guiping Zhong, Jiaming Zhou, Baoxiang Li

TL;DR

This paper proposes a novel model named CIF-Transducer (CIF-T) which incorporates the Continuous Integrate-and-Fire (Cif) mechanism with the RNN-T model to achieve efficient alignment and achieves state-of-the-art results with lower computational overhead compared to Rnn-T models.

Abstract

RNN-T models are widely used in ASR, which rely on the RNN-T loss to achieve length alignment between input audio and target sequence. However, the implementation complexity and the alignment-based optimization target of RNN-T loss lead to computational redundancy and a reduced role for predictor network, respectively. In this paper, we propose a novel model named CIF-Transducer (CIF-T) which incorporates the Continuous Integrate-and-Fire (CIF) mechanism with the RNN-T model to achieve efficient alignment. In this way, the RNN-T loss is abandoned, thus bringing a computational reduction and allowing the predictor network a more significant role. We also introduce Funnel-CIF, Context Blocks, Unified Gating and Bilinear Pooling joint network, and auxiliary training strategy to further improve performance. Experiments on the 178-hour AISHELL-1 and 10000-hour WenetSpeech datasets show that CIF-T achieves state-of-the-art results with lower computational overhead compared to RNN-T models.

CIF-T: A Novel CIF-based Transducer Architecture for Automatic Speech Recognition

TL;DR

This paper proposes a novel model named CIF-Transducer (CIF-T) which incorporates the Continuous Integrate-and-Fire (Cif) mechanism with the RNN-T model to achieve efficient alignment and achieves state-of-the-art results with lower computational overhead compared to Rnn-T models.

Abstract

RNN-T models are widely used in ASR, which rely on the RNN-T loss to achieve length alignment between input audio and target sequence. However, the implementation complexity and the alignment-based optimization target of RNN-T loss lead to computational redundancy and a reduced role for predictor network, respectively. In this paper, we propose a novel model named CIF-Transducer (CIF-T) which incorporates the Continuous Integrate-and-Fire (CIF) mechanism with the RNN-T model to achieve efficient alignment. In this way, the RNN-T loss is abandoned, thus bringing a computational reduction and allowing the predictor network a more significant role. We also introduce Funnel-CIF, Context Blocks, Unified Gating and Bilinear Pooling joint network, and auxiliary training strategy to further improve performance. Experiments on the 178-hour AISHELL-1 and 10000-hour WenetSpeech datasets show that CIF-T achieves state-of-the-art results with lower computational overhead compared to RNN-T models.
Paper Structure (13 sections, 7 equations, 2 figures, 6 tables)

This paper contains 13 sections, 7 equations, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. RNN Transducer
  4. Continuous Integrate-and-Fire
  5. CIF-Transducer
  6. Funnel-CIF and Context Blocks
  7. Unified Gating and Bilinear Pooling Joint Network
  8. Auxiliary Training Strategy
  9. Experiments
  10. Experimental Setup
  11. Results
  12. Ablation Study and Analysis
  13. Conclusion

Figures (2)

  • Figure 1: The different aggregation processes of acoustic features between RNN-T and CIF. The RNN-T emits special symbols $blank$ for the alignment process, while CIF aggregates the weighted $\alpha$ of acoustic features.
  • Figure 2: The structure of the proposed CIF-Transducer and Funnel-CIF. The dashed boxes in Fig. (a) represent the modules used only for the training process. FC and Conv stand for fully connected layer and convolutional layer, respectively.