Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BERT-LID: Leveraging BERT to Improve Spoken Language Identification

Yuting Nie, Junhong Zhao, Wei-Qiang Zhang, Jinfeng Bai

TL;DR

This work addresses the challenge of language identification for short speech segments and code-switching by introducing BERT-LID, a phonotactic-syntactic LID framework that uses phonetic posteriorgrams as BERT inputs and a deep classifier. By comparing various token embedding schemes and back-end classifiers, the authors show that phone-wise averaged PPG inputs with a RCNN classifier yield the strongest performance, outperforming traditional n-gram-SVM and x-vector baselines on OLR20, T&T, and TAL_ASR, with the largest gains on short segments. The results demonstrate the effectiveness of combining phonotactic representations and BERT-based language modeling for robust LID, particularly in real-world multilingual and code-switching scenarios. The approach provides a practical pathway to improve multilingual ASR and related systems by more accurately identifying languages in brief or mixed-language utterances.

Abstract

Language identification is the task of automatically determining the identity of a language conveyed by a spoken segment. It has a profound impact on the multilingual interoperability of an intelligent speech system. Despite language identification attaining high accuracy on medium or long utterances(>3s), the performance on short utterances (<=1s) is still far from satisfactory. We propose a BERT-based language identification system (BERT-LID) to improve language identification performance, especially on short-duration speech segments. We extend the original BERT model by taking the phonetic posteriorgrams (PPG) derived from the front-end phone recognizer as input. Then we deployed the optimal deep classifier followed by it for language identification. Our BERT-LID model can improve the baseline accuracy by about 6.5% on long-segment identification and 19.9% on short-segment identification, demonstrating our BERT-LID's effectiveness to language identification.

BERT-LID: Leveraging BERT to Improve Spoken Language Identification

TL;DR

This work addresses the challenge of language identification for short speech segments and code-switching by introducing BERT-LID, a phonotactic-syntactic LID framework that uses phonetic posteriorgrams as BERT inputs and a deep classifier. By comparing various token embedding schemes and back-end classifiers, the authors show that phone-wise averaged PPG inputs with a RCNN classifier yield the strongest performance, outperforming traditional n-gram-SVM and x-vector baselines on OLR20, T&T, and TAL_ASR, with the largest gains on short segments. The results demonstrate the effectiveness of combining phonotactic representations and BERT-based language modeling for robust LID, particularly in real-world multilingual and code-switching scenarios. The approach provides a practical pathway to improve multilingual ASR and related systems by more accurately identifying languages in brief or mixed-language utterances.

Abstract

Language identification is the task of automatically determining the identity of a language conveyed by a spoken segment. It has a profound impact on the multilingual interoperability of an intelligent speech system. Despite language identification attaining high accuracy on medium or long utterances(>3s), the performance on short utterances (<=1s) is still far from satisfactory. We propose a BERT-based language identification system (BERT-LID) to improve language identification performance, especially on short-duration speech segments. We extend the original BERT model by taking the phonetic posteriorgrams (PPG) derived from the front-end phone recognizer as input. Then we deployed the optimal deep classifier followed by it for language identification. Our BERT-LID model can improve the baseline accuracy by about 6.5% on long-segment identification and 19.9% on short-segment identification, demonstrating our BERT-LID's effectiveness to language identification.
Paper Structure (20 sections, 2 figures, 5 tables)

This paper contains 20 sections, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Language identification
  4. BERT
  5. BERT in language identification
  6. System Description
  7. Phone recognizer
  8. BERT adaption and token embedding methods
  9. Deep classifier variants
  10. Experiments
  11. Baseline systems
  12. Datasets
  13. Implementation detail
  14. Results
  15. Phone recognition impact
  16. ...and 5 more sections

Figures (2)

  • Figure 1: An example of an intra-sentential multilingual utterance, where the Chinese-English language interweaved in one speech segment. The total duration of the utterance is 1.9s, with the first Chinese part last 1s and the second English part last only 0.9s.
  • Figure 2: The scheme of BERT-LID. It features a customized BERT module to extract language-specific phonotactic representations and a deep classifier for language identification. In particular, we used the phone-wise averaged PPGs extracted from the phone recognizer as the token embeddings of BERT. We used RCNN lai2015recurrent as the deep classifier, following BERT.