Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Empowering Low-Resource Language ASR via Large-Scale Pseudo Labeling

Kaushal Santosh Bhogale, Deovrat Mehendale, Niharika Parasa, Sathish Kumar Reddy G, Tahir Javed, Pratyush Kumar, Mitesh M. Khapra

TL;DR

This work targets Hindi ASR under scarce labeled data by introducing Pratinidhi, a generic pseudo-labeling framework that combines multiple pseudo-transcribers and evaluators to produce high-quality transcripts for unlabeled audio. It leverages a consensus-based labeling mechanism and two evaluators (confidence-based and SONAR-based multimodal similarity) to filter transcripts, with a one-shot iteration used in practice. The authors validate the approach on IndicYT, a diverse YouTube-based Hindi benchmark, and demonstrate substantial domain-wide improvements (average ~8.6%) when augmenting training data with PN-pseudolab, while keeping out-of-domain performance stable. A new IndicYT benchmark, publicly available code, and models underpin the practical impact of scalable pseudo-labeling for low-resource languages.

Abstract

In this study, we tackle the challenge of limited labeled data for low-resource languages in ASR, focusing on Hindi. Specifically, we explore pseudo-labeling, by proposing a generic framework combining multiple ideas from existing works. Our framework integrates multiple base models for transcription and evaluators for assessing audio-transcript pairs, resulting in robust pseudo-labeling for low resource languages. We validate our approach with a new benchmark, IndicYT, comprising diverse YouTube audio files from multiple content categories. Our findings show that augmenting pseudo labeled data from YouTube with existing training data leads to significant performance improvements on IndicYT, without affecting performance on out-of-domain benchmarks, demonstrating the efficacy of pseudo-labeled data in enhancing ASR capabilities for low-resource languages. The benchmark, code and models developed as a part of this work will be made publicly available.

Empowering Low-Resource Language ASR via Large-Scale Pseudo Labeling

TL;DR

This work targets Hindi ASR under scarce labeled data by introducing Pratinidhi, a generic pseudo-labeling framework that combines multiple pseudo-transcribers and evaluators to produce high-quality transcripts for unlabeled audio. It leverages a consensus-based labeling mechanism and two evaluators (confidence-based and SONAR-based multimodal similarity) to filter transcripts, with a one-shot iteration used in practice. The authors validate the approach on IndicYT, a diverse YouTube-based Hindi benchmark, and demonstrate substantial domain-wide improvements (average ~8.6%) when augmenting training data with PN-pseudolab, while keeping out-of-domain performance stable. A new IndicYT benchmark, publicly available code, and models underpin the practical impact of scalable pseudo-labeling for low-resource languages.

Abstract

In this study, we tackle the challenge of limited labeled data for low-resource languages in ASR, focusing on Hindi. Specifically, we explore pseudo-labeling, by proposing a generic framework combining multiple ideas from existing works. Our framework integrates multiple base models for transcription and evaluators for assessing audio-transcript pairs, resulting in robust pseudo-labeling for low resource languages. We validate our approach with a new benchmark, IndicYT, comprising diverse YouTube audio files from multiple content categories. Our findings show that augmenting pseudo labeled data from YouTube with existing training data leads to significant performance improvements on IndicYT, without affecting performance on out-of-domain benchmarks, demonstrating the efficacy of pseudo-labeled data in enhancing ASR capabilities for low-resource languages. The benchmark, code and models developed as a part of this work will be made publicly available.
Paper Structure (10 sections, 3 equations, 1 figure, 4 tables)

This paper contains 10 sections, 3 equations, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Pratinidhi: A generic framework for large-scale pseudo-labeling
  4. The Framework
  5. Application to a Low-resource Indian Language
  6. PN-pseudolab
  7. IndicYT Benchmark
  8. Results and Discussion
  9. Conclusion
  10. Acknowledgements

Figures (1)

  • Figure 1: Comparison of our model on the IndicYT benchmark shows improvement across all domains, with significant improvement on Education domains like Math and Science.