Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Effective and Efficient Non-autoregressive Decoding Using Block-based Attention Mask

Tianzi Wang, Xurong Xie, Zhaoqing Li, Shoukang Hu, Zengrui Jin, Jiajun Deng, Mingyu Cui, Shujie Hu, Mengzhe Geng, Guinan Li, Helen Meng, Xunying Liu

TL;DR

A novel non-autoregressive (NAR) block-based Attention Mask Decoder (AMD) that flexibly balances performance-efficiency trade-offs for Conformer ASR systems is proposed.

Abstract

This paper proposes a novel non-autoregressive (NAR) block-based Attention Mask Decoder (AMD) that flexibly balances performance-efficiency trade-offs for Conformer ASR systems. AMD performs parallel NAR inference within contiguous blocks of output labels that are concealed using attention masks, while conducting left-to-right AR prediction and history context amalgamation between blocks. A beam search algorithm is designed to leverage a dynamic fusion of CTC, AR Decoder, and AMD probabilities. Experiments on the LibriSpeech-100hr corpus suggest the tripartite Decoder incorporating the AMD module produces a maximum decoding speed-up ratio of 1.73x over the baseline CTC+AR decoding, while incurring no statistically significant word error rate (WER) increase on the test sets. When operating with the same decoding real time factors, statistically significant WER reductions of up to 0.7% and 0.3% absolute (5.3% and 6.1% relative) were obtained over the CTC+AR baseline.

Towards Effective and Efficient Non-autoregressive Decoding Using Block-based Attention Mask

TL;DR

A novel non-autoregressive (NAR) block-based Attention Mask Decoder (AMD) that flexibly balances performance-efficiency trade-offs for Conformer ASR systems is proposed.

Abstract

This paper proposes a novel non-autoregressive (NAR) block-based Attention Mask Decoder (AMD) that flexibly balances performance-efficiency trade-offs for Conformer ASR systems. AMD performs parallel NAR inference within contiguous blocks of output labels that are concealed using attention masks, while conducting left-to-right AR prediction and history context amalgamation between blocks. A beam search algorithm is designed to leverage a dynamic fusion of CTC, AR Decoder, and AMD probabilities. Experiments on the LibriSpeech-100hr corpus suggest the tripartite Decoder incorporating the AMD module produces a maximum decoding speed-up ratio of 1.73x over the baseline CTC+AR decoding, while incurring no statistically significant word error rate (WER) increase on the test sets. When operating with the same decoding real time factors, statistically significant WER reductions of up to 0.7% and 0.3% absolute (5.3% and 6.1% relative) were obtained over the CTC+AR baseline.
Paper Structure (8 sections, 8 equations, 4 figures, 2 tables)

This paper contains 8 sections, 8 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Hybrid Attention Encoder-Decoder Based ASR System
  3. Block-Based Attention Mask Decoder
  4. Block-Based Attention-Masking Decoder
  5. Beam Search Using AMD
  6. Experiments
  7. Conclusions
  8. Acknowledgements

Figures (4)

  • Figure 1: The proposed Conformer ASR system architecture using a tripartite Decoder that includes the proposed non-autoregressive block-based attention mask decoder (AMD) (left, dashed yellow line) in addition to the CTC module (centre, purple) and attention-based AR Decoder (right, dashed blue line).
  • Figure 2: Inference using: a) an AR Decoder, b) a Mask Prediction Decoder; c) the proposed AMD. 'msk' refers to input mask tokens, M is the attention mask within contiguous blocks for parallel inference via AMD, 'B' refers to the block size. Tokens highlighted in green background denote Decoder inputs at the current inference step. Tokens highlighted in purple denote predicted tokens at current step. Tokens in dashed boxes in a) and c) represent those from the previous inference step. Tokens with "_" denote those obtained from CTC prediction.
  • Figure 3: AMD lattice density and oracle WERs computed using 100-best hypotheses over varying top-K beam size from 1 to 20.
  • Figure :