Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Advancing Arabic Speech Recognition Through Large-Scale Weakly Supervised Learning

Mahmoud Salhab, Marwan Elghitany, Shameed Sait, Syed Sibghat Ullah, Mohammad Abusheikh, Hasan Abusheikh

TL;DR

This work tackles Arabic ASR under data scarcity by applying weakly supervised learning to train a Conformer-based model from 15,000 hours of weakly annotated data spanning Modern Standard Arabic and Dialectal Arabic. A multi-stage weak label generation pipeline uses segmentation, hypothesis generation, Levenshtein-based agreement, and perplexity filtering to create usable training signals without manual transcriptions. The model, trained with CTC loss and a 1024-token SentencePiece, achieves state-of-the-art results on standard Arabic benchmarks, substantially outperforming open- and closed-source baselines and reducing WER and CER relative to prior best models. The approach demonstrates a scalable, cost-efficient path to high-performance ASR in low-resource language settings and may generalize to other languages with similar data constraints.

Abstract

Automatic speech recognition (ASR) is crucial for human-machine interaction in diverse applications like conversational agents, industrial robotics, call center automation, and automated subtitling. However, developing high-performance ASR models remains challenging, particularly for low-resource languages like Arabic, due to the scarcity of large, labeled speech datasets, which are costly and labor-intensive to produce. In this work, we employ weakly supervised learning to train an Arabic ASR model using the Conformer architecture. Our model is trained from scratch on 15,000 hours of weakly annotated speech data covering both Modern Standard Arabic (MSA) and Dialectal Arabic (DA), eliminating the need for costly manual transcriptions. Despite the absence of human-verified labels, our approach achieves state-of-the-art (SOTA) results in Arabic ASR, surpassing both open and closed-source models on standard benchmarks. By demonstrating the effectiveness of weak supervision as a scalable, cost-efficient alternative to traditional supervised approaches, paving the way for improved ASR systems in low resource settings.

Advancing Arabic Speech Recognition Through Large-Scale Weakly Supervised Learning

TL;DR

This work tackles Arabic ASR under data scarcity by applying weakly supervised learning to train a Conformer-based model from 15,000 hours of weakly annotated data spanning Modern Standard Arabic and Dialectal Arabic. A multi-stage weak label generation pipeline uses segmentation, hypothesis generation, Levenshtein-based agreement, and perplexity filtering to create usable training signals without manual transcriptions. The model, trained with CTC loss and a 1024-token SentencePiece, achieves state-of-the-art results on standard Arabic benchmarks, substantially outperforming open- and closed-source baselines and reducing WER and CER relative to prior best models. The approach demonstrates a scalable, cost-efficient path to high-performance ASR in low-resource language settings and may generalize to other languages with similar data constraints.

Abstract

Automatic speech recognition (ASR) is crucial for human-machine interaction in diverse applications like conversational agents, industrial robotics, call center automation, and automated subtitling. However, developing high-performance ASR models remains challenging, particularly for low-resource languages like Arabic, due to the scarcity of large, labeled speech datasets, which are costly and labor-intensive to produce. In this work, we employ weakly supervised learning to train an Arabic ASR model using the Conformer architecture. Our model is trained from scratch on 15,000 hours of weakly annotated speech data covering both Modern Standard Arabic (MSA) and Dialectal Arabic (DA), eliminating the need for costly manual transcriptions. Despite the absence of human-verified labels, our approach achieves state-of-the-art (SOTA) results in Arabic ASR, surpassing both open and closed-source models on standard benchmarks. By demonstrating the effectiveness of weak supervision as a scalable, cost-efficient alternative to traditional supervised approaches, paving the way for improved ASR systems in low resource settings.

Paper Structure

This paper contains 11 sections, 9 equations, 1 figure, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Weakly Supervised Learning
  5. Generation of Weak Labels
  6. Model Architecture
  7. Experiments and Results
  8. Experimental Setup
  9. Evaluation Metrics & Datasets
  10. Results
  11. Conclusion

Figures (1)

  • Figure 1: Illustration of the weak label generation process, which includes speech processing, generating multiple hypotheses, selecting the most probable hypotheses, and segments merging.