Conformer-1: Robust ASR via Large-Scale Semisupervised Bootstrapping

TL;DR

Conformer-1 addresses the data scarcity challenge in ASR by applying large-scale semisupervised bootstrapping, generating pseudo-labels for a substantial unlabeled public audio corpus and training dual ASR models (Async and Realtime) on a roughly 570k-hour dataset. By combining 57k hours of labeled data with up to 520k hours of pseudo-labeled data, the approach yields significant relative $WER$ reductions (11.5% for Async and 24.3% for Realtime) and improved noise robustness, while introducing a Proper Noun accuracy metric for more human-aligned evaluation. The work shows Conformer-1 surpasses Whisper and other providers on many benchmarks and demonstrates robustness to Gaussian and ambient noise as data scales, albeit with some domain-specific limitations. Overall, the paper argues for semi-supervised data scaling as a practical path to state-of-the-art ASR performance with reduced labeling requirements and highlights directions for better pseudo-labeling, punctuation integration, and evaluation metrics.

Abstract

This paper presents Conformer-1, an end-to-end Automatic Speech Recognition (ASR) model trained on an extensive dataset of 570k hours of speech audio data, 91% of which was acquired from publicly available sources. To achieve this, we perform Noisy Student Training after generating pseudo-labels for the unlabeled public data using a strong Conformer RNN-T baseline model. The addition of these pseudo-labeled data results in remarkable improvements in relative Word Error Rate (WER) by 11.5% and 24.3% for our asynchronous and realtime models, respectively. Additionally, the model is more robust to background noise owing to the addition of these data. The results obtained in this study demonstrate that the incorporation of pseudo-labeled publicly available data is a highly effective strategy for improving ASR accuracy and noise robustness.

Figures (6)

• Figure 1: Progressive left-context receptive field of Streaming Transducer, which grows in accordance with depth only on the left side.
• Figure 2: Conformer-1 outperforms Whisper and four other providers in most of our in-house, private benchmarks. The distribution of WER from six ASR systems on six in-house, private benchmarks are compared. The boxes show the quartiles of per-example WERs, and the per-dataset aggregate WERs are annotated on each box. The one benchmarks where Conformer-1 does not outperfrom every other model is heavy in numbers, which is an area that we will improve in our next generation of ASR models.
• Figure 3: Conformer-1 results in lower WER than Whisper for eight out of eleven open source benchmarks. Conformer-1 also outperforms other two providers in seven out of 11 open source benchmarks.
• Figure 4: Conformer-1 performance on Proper Nouns as measured by WER and Jaro-Winkler metrics. Only two datasets were used in this analysis because they were the only datasets with punctuated and cased true labels. Conformer-1 does not outperform Whisper on Proper Noun data.
• Figure 5: Noise robustness with increase in amount of pseudo labeled data