SlimIPL: Language-Model-Free Iterative Pseudo-Labeling

TL;DR

The paper tackles the reliance on language models for pseudo-label generation in semi-supervised ASR by introducing slimIPL, a language-model-free iterative pseudo-labeling method that uses a dynamic cache of hard labels produced by the acoustic model. It demonstrates that this LM-free approach can achieve competitive, state-of-the-art performance in low-resource LibriSpeech settings and even surpass LM-free baselines in 100 hours of labeled data, while drastically reducing computational requirements. The method also extends to conversational speech, showing broad applicability and robustness across domains. Overall, slimIPL simplifies semi-supervised ASR training, reduces the risk of LM overfitting, and delivers practical speedups without sacrificing accuracy.

Abstract

Recent results in end-to-end automatic speech recognition have demonstrated the efficacy of pseudo-labeling for semi-supervised models trained both with Connectionist Temporal Classification (CTC) and Sequence-to-Sequence (seq2seq) losses. Iterative Pseudo-Labeling (IPL), which continuously trains a single model using pseudo-labels iteratively re-generated as the model learns, has been shown to further improve performance in ASR. We improve upon the IPL algorithm: as the model learns, we propose to iteratively re-generate transcriptions with hard labels (the most probable tokens), that is, without a language model. We call this approach Language-Model-Free IPL (slimIPL) and give a resultant training setup for low-resource settings with CTC-based models. slimIPL features a dynamic cache for pseudo-labels which reduces sensitivity to changes in relabeling hyperparameters and results in improves training stability. slimIPL is also highly-efficient and requires 3.5-4x fewer computational resources to converge than other state-of-the-art semi/self-supervised approaches. With only 10 hours of labeled audio, slimIPL is competitive with self-supervised approaches, and is state-of-the-art with 100 hours of labeled audio without the use of a language model both at test time and during pseudo-label generation.

Figures (1)

• Figure 1: Learning curves on dev-other for models trained on LL-10/LS-960 (left) and LS-100/LS-860 (right). slimIPL models refer to baseline models (grey) from Table \ref{['tab:ablations']}.