Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Automatic Speech Recognition Advancements for Indigenous Languages of the Americas

Monica Romero, Sandra Gomez, Ivan G. Torre

TL;DR

The fine-tuning of a state-of-the-art ASR model for each target language, using approximately 36.65 h of transcribed speech data from diverse sources enriched with data augmentation methods, is described, marking the first open ASR models for Wa’ikhana and Kotiria.

Abstract

Indigenous languages are a fundamental legacy in the development of human communication, embodying the unique identity and culture of local communities in America. The Second AmericasNLP (Americas Natural Language Processing) Competition Track 1 of NeurIPS (Neural Information Processing Systems) 2022 proposed the task of training automatic speech recognition (ASR) systems for five Indigenous languages: Quechua, Guarani, Bribri, Kotiria, and Wa'ikhana. In this paper, we describe the fine-tuning of a state-of-the-art ASR model for each target language, using approximately 36.65 h of transcribed speech data from diverse sources enriched with data augmentation methods. We systematically investigate, using a Bayesian search, the impact of the different hyperparameters on the Wav2vec2.0 XLS-R (Cross-Lingual Speech Representations) variants of 300 M and 1 B parameters. Our findings indicate that data and detailed hyperparameter tuning significantly affect ASR accuracy, but language complexity determines the final result. The Quechua model achieved the lowest character error rate (CER) (12.14), while the Kotiria model, despite having the most extensive dataset during the fine-tuning phase, showed the highest CER (36.59). Conversely, with the smallest dataset, the Guarani model achieved a CER of 15.59, while Bribri and Wa'ikhana obtained, respectively, CERs of 34.70 and 35.23. Additionally, Sobol' sensitivity analysis highlighted the crucial roles of freeze fine-tuning updates and dropout rates. We release our best models for each language, marking the first open ASR models for Wa'ikhana and Kotiria. This work opens avenues for future research to advance ASR techniques in preserving minority Indigenous languages

Automatic Speech Recognition Advancements for Indigenous Languages of the Americas

TL;DR

The fine-tuning of a state-of-the-art ASR model for each target language, using approximately 36.65 h of transcribed speech data from diverse sources enriched with data augmentation methods, is described, marking the first open ASR models for Wa’ikhana and Kotiria.

Abstract

Indigenous languages are a fundamental legacy in the development of human communication, embodying the unique identity and culture of local communities in America. The Second AmericasNLP (Americas Natural Language Processing) Competition Track 1 of NeurIPS (Neural Information Processing Systems) 2022 proposed the task of training automatic speech recognition (ASR) systems for five Indigenous languages: Quechua, Guarani, Bribri, Kotiria, and Wa'ikhana. In this paper, we describe the fine-tuning of a state-of-the-art ASR model for each target language, using approximately 36.65 h of transcribed speech data from diverse sources enriched with data augmentation methods. We systematically investigate, using a Bayesian search, the impact of the different hyperparameters on the Wav2vec2.0 XLS-R (Cross-Lingual Speech Representations) variants of 300 M and 1 B parameters. Our findings indicate that data and detailed hyperparameter tuning significantly affect ASR accuracy, but language complexity determines the final result. The Quechua model achieved the lowest character error rate (CER) (12.14), while the Kotiria model, despite having the most extensive dataset during the fine-tuning phase, showed the highest CER (36.59). Conversely, with the smallest dataset, the Guarani model achieved a CER of 15.59, while Bribri and Wa'ikhana obtained, respectively, CERs of 34.70 and 35.23. Additionally, Sobol' sensitivity analysis highlighted the crucial roles of freeze fine-tuning updates and dropout rates. We release our best models for each language, marking the first open ASR models for Wa'ikhana and Kotiria. This work opens avenues for future research to advance ASR techniques in preserving minority Indigenous languages
Paper Structure (11 sections, 2 figures, 4 tables)

This paper contains 11 sections, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Experimental Setup and Dataset Description
  4. Main Architecture and Pre-Trained Models
  5. Data
  6. Decoding Strategy and Language Models
  7. Hyperparameter Fine-Tuning
  8. Sobol' Sensitivity Analysis for Hyperparameter Explanation
  9. Results
  10. Conclusions and Future Work
  11. Data and Models Accessibility

Figures (2)

  • Figure S1: Sketch of the dataset used for fine-tuning the ASR system, the CNN and transformer-based architecture wav2vec2.0, the fine-tuning process, the Bayesian hyperparameter search and the Sobol sensitivity analysis.
  • Figure S2: The outer bar chart panel displays the character error rates (CERs) for five Indigenous language models: Kotiria, Wa'ikhana, Bribri, Guarani, and Quechua. Lower bars indicate better-quality performance of the model. The inner panel provides a Sobol' sensitivity analysis of the various hyperparameters tuned during model training, assessing their impact on model performance variability. The orange bars represent the total sensitivity (ST) index, while the green bars indicate the first-order sensitivity (S1) index. A higher bar indicates the more importance of that hyperparameter when correctly choosing it during the fine-tuning phase.