Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Semantically Corrected Amharic Automatic Speech Recognition

Samuael Adnew, Paul Pu Liang

TL;DR

This work tackles semantic errors in Amharic ASR caused by spacing in the Ge\'ez script by combining two components: fine-tuning XLSR-Wav2Vec2-based ASR models for Amharic at the phoneme and character levels, and a Seq2Seq transformer that post-processes ASR outputs into semantically coherent Amharic sentences. The authors release corrected Amharic test transcriptions and demonstrate substantial improvements over baselines, achieving a Character Error Rate of $5.5\%$ and a Word Error Rate of $23.3\%$ on corrected data. The approach highlights the value of semantic post-processing for low-resource languages and offers a modular framework that can extend to other Ethiopic-script languages, with code and models available on GitHub. The work also underscores the importance of dataset corrections when evaluating Amharic ASR in real-world settings.

Abstract

Automatic Speech Recognition (ASR) can play a crucial role in enhancing the accessibility of spoken languages worldwide. In this paper, we build a set of ASR tools for Amharic, a language spoken by more than 50 million people primarily in eastern Africa. Amharic is written in the Ge'ez script, a sequence of graphemes with spacings denoting word boundaries. This makes computational processing of Amharic challenging since the location of spacings can significantly impact the meaning of formed sentences. We find that existing benchmarks for Amharic ASR do not account for these spacings and only measure individual grapheme error rates, leading to significantly inflated measurements of in-the-wild performance. In this paper, we first release corrected transcriptions of existing Amharic ASR test datasets, enabling the community to accurately evaluate progress. Furthermore, we introduce a post-processing approach using a transformer encoder-decoder architecture to organize raw ASR outputs into a grammatically complete and semantically meaningful Amharic sentence. Through experiments on the corrected test dataset, our model enhances the semantic correctness of Amharic speech recognition systems, achieving a Character Error Rate (CER) of 5.5\% and a Word Error Rate (WER) of 23.3\%.

Semantically Corrected Amharic Automatic Speech Recognition

TL;DR

This work tackles semantic errors in Amharic ASR caused by spacing in the Ge\'ez script by combining two components: fine-tuning XLSR-Wav2Vec2-based ASR models for Amharic at the phoneme and character levels, and a Seq2Seq transformer that post-processes ASR outputs into semantically coherent Amharic sentences. The authors release corrected Amharic test transcriptions and demonstrate substantial improvements over baselines, achieving a Character Error Rate of and a Word Error Rate of on corrected data. The approach highlights the value of semantic post-processing for low-resource languages and offers a modular framework that can extend to other Ethiopic-script languages, with code and models available on GitHub. The work also underscores the importance of dataset corrections when evaluating Amharic ASR in real-world settings.

Abstract

Automatic Speech Recognition (ASR) can play a crucial role in enhancing the accessibility of spoken languages worldwide. In this paper, we build a set of ASR tools for Amharic, a language spoken by more than 50 million people primarily in eastern Africa. Amharic is written in the Ge'ez script, a sequence of graphemes with spacings denoting word boundaries. This makes computational processing of Amharic challenging since the location of spacings can significantly impact the meaning of formed sentences. We find that existing benchmarks for Amharic ASR do not account for these spacings and only measure individual grapheme error rates, leading to significantly inflated measurements of in-the-wild performance. In this paper, we first release corrected transcriptions of existing Amharic ASR test datasets, enabling the community to accurately evaluate progress. Furthermore, we introduce a post-processing approach using a transformer encoder-decoder architecture to organize raw ASR outputs into a grammatically complete and semantically meaningful Amharic sentence. Through experiments on the corrected test dataset, our model enhances the semantic correctness of Amharic speech recognition systems, achieving a Character Error Rate (CER) of 5.5\% and a Word Error Rate (WER) of 23.3\%.
Paper Structure (28 sections, 2 figures, 2 tables)

This paper contains 28 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Transformer-based Language Models for ASR
  4. Post-correction ASR
  5. ASR for Amharic and other low-resource languages
  6. Methodology
  7. Finetuning Wav2Vec ASR model for Amharic
  8. Data curation
  9. Training details
  10. Seq2Seq sentence correction model dataset and training
  11. Data curation
  12. Training details
  13. Results
  14. Baselines
  15. Main results
  16. ...and 13 more sections

Figures (2)

  • Figure 1: This figure illustrates the process facilitated by the Seq2Seq sentence correction model, demonstrating the restructuring of nonsensical sub-words into coherent and meaningful Amharic word sequences, culminating in the formation of a coherent sentence. Through this formal analysis, witness the progression from disorder to comprehension, ultimately leading to the generation of a refined and intelligible final output.
  • Figure 2: Our method comprises two parts: (1) the finetuned ASR model, which adapts base ASR models for Amharic, and (2) the Seq2Seq sentence correction model, designed to semantically correct transcribed Amharic outputs. In this diagram, the character 'x' is employed to denote meaningless sub-words.