Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

One-pass Multiple Conformer and Foundation Speech Systems Compression and Quantization Using An All-in-one Neural Model

Zhaoqing Li, Haoning Xu, Tianzi Wang, Shoukang Hu, Zengrui Jin, Shujie Hu, Jiajun Deng, Mingyu Cui, Mengzhe Geng, Xunying Liu

TL;DR

This work tackles the need for fine-grained, device-aware ASR compression by introducing a one-pass all-in-one neural framework that jointly compresses and quantizes multiple nested Conformer and wav2vec2.0 systems. It uses weight-sharing across sub-networks and a multitask training objective with optional KL regularization to align outputs, enabling depth, width, and precision variations within a single model. Across Switchboard and LibriSpeech, the all-in-one approach achieves comparable or better WER than independently trained baselines, with substantial compression (up to $12.8\times$ for Conformer and $3.93\times$ for wav2vec2.0) and significant training-time speed-ups (up to $3.4\times$). The results demonstrate robust performance across single- and multi-attribute compression, offering practical gains for deploying multiple ASR configurations on diverse devices. Future work may pursue more flexible, layer-wise reconfigurability to further tailor architectures to specific deployment constraints.

Abstract

We propose a novel one-pass multiple ASR systems joint compression and quantization approach using an all-in-one neural model. A single compression cycle allows multiple nested systems with varying Encoder depths, widths, and quantization precision settings to be simultaneously constructed without the need to train and store individual target systems separately. Experiments consistently demonstrate the multiple ASR systems compressed in a single all-in-one model produced a word error rate (WER) comparable to, or lower by up to 1.01\% absolute (6.98\% relative) than individually trained systems of equal complexity. A 3.4x overall system compression and training time speed-up was achieved. Maximum model size compression ratios of 12.8x and 3.93x were obtained over the baseline Switchboard-300hr Conformer and LibriSpeech-100hr fine-tuned wav2vec2.0 models, respectively, incurring no statistically significant WER increase.

One-pass Multiple Conformer and Foundation Speech Systems Compression and Quantization Using An All-in-one Neural Model

TL;DR

This work tackles the need for fine-grained, device-aware ASR compression by introducing a one-pass all-in-one neural framework that jointly compresses and quantizes multiple nested Conformer and wav2vec2.0 systems. It uses weight-sharing across sub-networks and a multitask training objective with optional KL regularization to align outputs, enabling depth, width, and precision variations within a single model. Across Switchboard and LibriSpeech, the all-in-one approach achieves comparable or better WER than independently trained baselines, with substantial compression (up to for Conformer and for wav2vec2.0) and significant training-time speed-ups (up to ). The results demonstrate robust performance across single- and multi-attribute compression, offering practical gains for deploying multiple ASR configurations on diverse devices. Future work may pursue more flexible, layer-wise reconfigurability to further tailor architectures to specific deployment constraints.

Abstract

We propose a novel one-pass multiple ASR systems joint compression and quantization approach using an all-in-one neural model. A single compression cycle allows multiple nested systems with varying Encoder depths, widths, and quantization precision settings to be simultaneously constructed without the need to train and store individual target systems separately. Experiments consistently demonstrate the multiple ASR systems compressed in a single all-in-one model produced a word error rate (WER) comparable to, or lower by up to 1.01\% absolute (6.98\% relative) than individually trained systems of equal complexity. A 3.4x overall system compression and training time speed-up was achieved. Maximum model size compression ratios of 12.8x and 3.93x were obtained over the baseline Switchboard-300hr Conformer and LibriSpeech-100hr fine-tuned wav2vec2.0 models, respectively, incurring no statistically significant WER increase.
Paper Structure (12 sections, 4 equations, 1 figure, 4 tables)

This paper contains 12 sections, 4 equations, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Conformer AED System and Wav2vec2.0 Foundation Model
  3. One-Pass Multiple Systems Compression
  4. All-In-One Weight-Sharing Neural Model
  5. Multiple Weight-Sharing Systems Training
  6. Kullback–Leibler Divergence Regularization
  7. Experiments
  8. Experimental Setup
  9. Results of Conformer Systems
  10. Results for wav2vec2.0 Systems
  11. Conclusion
  12. Acknowledgements

Figures (1)

  • Figure 1: Diagram of an all-in-one ASR Encoder. A partly colored area denotes a sub-network, and all sub-networks share weights with larger ones. (a) Sub-networks can skip top layers. (b) Sub-networks can mask partial FFN weight matrices. (c) Sub-networks can be directly quantized to a lower bit-width.