Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reduce, Reuse, Recycle: Is Perturbed Data better than Other Language augmentation for Low Resource Self-Supervised Speech Models

Asad Ullah, Alessandro Ragano, Andrew Hines

TL;DR

This paper tackles the data scarcity challenge in self-supervised speech representation learning (SSRL) for low-resource languages by comparing cross-lingual pre-training with artificial data augmentation during pre-training. Using an APC-based model trained on a 25-hour English proxy, the study examines pitch-only, noise-only, and combined pitch+noise augmentations versus adding other languages (African-accented English, Mandarin) and evaluates on phoneme recognition. The results show that combined pitch and noise augmentation delivers the strongest downstream gains, outperforming accent and cross-lingual transfer, and that roughly 17× augmentation is required to match performance achieved with target-language data. The findings suggest augmentation as a practical strategy to bolster SSRL in resource-constrained settings and highlight potential synergy with cross-lingual pre-training, with future work including TTS-based augmentation and broader open-source model applications.

Abstract

Self-supervised representation learning (SSRL) has demonstrated superior performance than supervised models for tasks including phoneme recognition. Training SSRL models poses a challenge for low-resource languages where sufficient pre-training data may not be available. A common approach is cross-lingual pre-training. Instead, we propose to use audio augmentation techniques, namely: pitch variation, noise addition, accented target language and other language speech to pre-train SSRL models in a low resource condition and evaluate phoneme recognition. Our comparisons found that a combined synthetic augmentations (noise/pitch) strategy outperformed accent and language knowledge transfer. Furthermore, we examined the scaling factor of augmented data to achieve equivalent performance to model pre-trained with target domain speech. Our findings suggest that for resource-constrained languages, combined augmentations can be a viable option than other augmentations.

Reduce, Reuse, Recycle: Is Perturbed Data better than Other Language augmentation for Low Resource Self-Supervised Speech Models

TL;DR

This paper tackles the data scarcity challenge in self-supervised speech representation learning (SSRL) for low-resource languages by comparing cross-lingual pre-training with artificial data augmentation during pre-training. Using an APC-based model trained on a 25-hour English proxy, the study examines pitch-only, noise-only, and combined pitch+noise augmentations versus adding other languages (African-accented English, Mandarin) and evaluates on phoneme recognition. The results show that combined pitch and noise augmentation delivers the strongest downstream gains, outperforming accent and cross-lingual transfer, and that roughly 17× augmentation is required to match performance achieved with target-language data. The findings suggest augmentation as a practical strategy to bolster SSRL in resource-constrained settings and highlight potential synergy with cross-lingual pre-training, with future work including TTS-based augmentation and broader open-source model applications.

Abstract

Self-supervised representation learning (SSRL) has demonstrated superior performance than supervised models for tasks including phoneme recognition. Training SSRL models poses a challenge for low-resource languages where sufficient pre-training data may not be available. A common approach is cross-lingual pre-training. Instead, we propose to use audio augmentation techniques, namely: pitch variation, noise addition, accented target language and other language speech to pre-train SSRL models in a low resource condition and evaluate phoneme recognition. Our comparisons found that a combined synthetic augmentations (noise/pitch) strategy outperformed accent and language knowledge transfer. Furthermore, we examined the scaling factor of augmented data to achieve equivalent performance to model pre-trained with target domain speech. Our findings suggest that for resource-constrained languages, combined augmentations can be a viable option than other augmentations.
Paper Structure (11 sections, 3 figures, 2 tables)

This paper contains 11 sections, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Proposed Method
  3. Experimental Setup
  4. Datasets
  5. APC Architecture
  6. Experiments
  7. Results
  8. Comparison of Augmentation Strategies
  9. Augmentation Size Analysis
  10. Conclusions
  11. Acknowledgements

Figures (3)

  • Figure 1: Linguistic Distance from American English
  • Figure 2: Performance of 100-hrs augmented pre-training data (25-clean-hrs + 75-aug-hrs) compared to the baseline 25-clean-hrs (baseline acc. 51.5%)
  • Figure 3: Performance of augmented vs clean data in pre-training