Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ViSpeechFormer: A Phonemic Approach for Vietnamese Automatic Speech Recognition

Khoa Anh Nguyen, Long Minh Hoang, Nghia Hieu Nguyen, Luan Thanh Nguyen, Ngan Luu-Thuy Nguyen

TL;DR

This work tackles Vietnamese ASR by exploiting the language's phonetic orthography to perform phoneme-level decoding. It introduces ViPhonER, a Vietnamese Phonemic Tokenization Algorithm, and ViSpeechFormer, a Speech Transformer equipped with a three-headed phonemic decoder that outputs initials, rhymes, and tones as IPA phonemes before detokenization. Across ViVOS and LSVSC datasets, the phoneme-based approach achieves state-of-the-art CER/WER and notable improvements in handling OOV words, while maintaining a compact decoder and efficient inference. The results suggest that explicit phoneme modeling is particularly advantageous for tonal, isolating languages with sparse morphology and may generalize to other phonetic orthographies.

Abstract

Vietnamese has a phonetic orthography, where each grapheme corresponds to at most one phoneme and vice versa. Exploiting this high grapheme-phoneme transparency, we propose ViSpeechFormer (\textbf{Vi}etnamese \textbf{Speech} Trans\textbf{Former}), a phoneme-based approach for Vietnamese Automatic Speech Recognition (ASR). To the best of our knowledge, this is the first Vietnamese ASR framework that explicitly models phonemic representations. Experiments on two publicly available Vietnamese ASR datasets show that ViSpeechFormer achieves strong performance, generalizes better to out-of-vocabulary words, and is less affected by training bias. This phoneme-based paradigm is also promising for other languages with phonetic orthographies. The code will be released upon acceptance of this paper.

ViSpeechFormer: A Phonemic Approach for Vietnamese Automatic Speech Recognition

TL;DR

This work tackles Vietnamese ASR by exploiting the language's phonetic orthography to perform phoneme-level decoding. It introduces ViPhonER, a Vietnamese Phonemic Tokenization Algorithm, and ViSpeechFormer, a Speech Transformer equipped with a three-headed phonemic decoder that outputs initials, rhymes, and tones as IPA phonemes before detokenization. Across ViVOS and LSVSC datasets, the phoneme-based approach achieves state-of-the-art CER/WER and notable improvements in handling OOV words, while maintaining a compact decoder and efficient inference. The results suggest that explicit phoneme modeling is particularly advantageous for tonal, isolating languages with sparse morphology and may generalize to other phonetic orthographies.

Abstract

Vietnamese has a phonetic orthography, where each grapheme corresponds to at most one phoneme and vice versa. Exploiting this high grapheme-phoneme transparency, we propose ViSpeechFormer (\textbf{Vi}etnamese \textbf{Speech} Trans\textbf{Former}), a phoneme-based approach for Vietnamese Automatic Speech Recognition (ASR). To the best of our knowledge, this is the first Vietnamese ASR framework that explicitly models phonemic representations. Experiments on two publicly available Vietnamese ASR datasets show that ViSpeechFormer achieves strong performance, generalizes better to out-of-vocabulary words, and is less affected by training bias. This phoneme-based paradigm is also promising for other languages with phonetic orthographies. The code will be released upon acceptance of this paper.
Paper Structure (17 sections, 13 equations, 4 figures, 6 tables, 2 algorithms)

This paper contains 17 sections, 13 equations, 4 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology
  4. Backgrounds
  5. ViPhonER - Vietnamese Phonemic Tokenization Algorithm
  6. ViSpeechFormer - Vietnamese Speech Transformer
  7. Objective Function
  8. Experiments
  9. Datasets, Metrics and Baselines
  10. Configuration
  11. Data Preparation
  12. Experimental Results
  13. Experimental Analysis
  14. Analysis on Ability of Giving Correct Words
  15. Analysis on Giving Correct Out-of-Vocab Words
  16. ...and 2 more sections

Figures (4)

  • Figure 1: The structure of syllable in Vietnamese.
  • Figure 2: Example of the transparent between graphemes and phonemes in Vietnamese. The given word in this example means royal in English.
  • Figure 3: The architecture of the proposed ViSpeechFormer method. FC stands for Fully Connected Layer. FFN stands for Feed-Forward Network.
  • Figure 4: The architecture of Feed-Forward Network.