Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Parameter-efficient Language Extension Framework for Multilingual ASR

Wei Liu, Jingyong Hou, Dong Yang, Muyong Cao, Tan Lee

TL;DR

This work tackles the problem of extending a multilingual ASR (MASR) to new languages without catastrophic forgetting by decomposing continual learning into language identity prediction (LP) and cross-lingual adaptation (XLA). It introduces PELE, an architecture-based, parameter-efficient framework that uses language-specific add-on PEFT modules to adapt to new languages while freezing the base MASR. Empirical results across five low-resource languages show that Adapter-based XLA within PELE yields the strongest performance, outperforming several baselines and approaching the continual joint training upper bound with far fewer parameters. The findings highlight the importance of using module-based, adapter-style approaches over weight- or input-focused PEFTs for MASR language extension, and they demonstrate the practical potential for scalable multilingual extension.

Abstract

Covering all languages with a multilingual speech recognition model (MASR) is very difficult. Performing language extension on top of an existing MASR is a desirable choice. In this study, the MASR continual learning problem is probabilistically decomposed into language identity prediction (LP) and cross-lingual adaptation (XLA) sub-problems. Based on this, we propose an architecture-based framework for language extension that can fundamentally solve catastrophic forgetting, debudded as PELE. PELE is designed to be parameter-efficient, incrementally incorporating an add-on module to adapt to a new language. Specifically, different parameter-efficient fine-tuning (PEFT) modules and their variants are explored as potential candidates to perform XLA. Experiments are carried out on 5 new languages with a wide range of low-resourced data sizes. The best-performing PEFT candidate can achieve satisfactory performance across all languages and demonstrates superiority in three of five languages over the continual joint learning setting. Notably, PEFT methods focusing on weight parameters or input features are revealed to be limited in performance, showing significantly inferior extension capabilities compared to inserting a lightweight module in between layers such as an Adapter.

A Parameter-efficient Language Extension Framework for Multilingual ASR

TL;DR

This work tackles the problem of extending a multilingual ASR (MASR) to new languages without catastrophic forgetting by decomposing continual learning into language identity prediction (LP) and cross-lingual adaptation (XLA). It introduces PELE, an architecture-based, parameter-efficient framework that uses language-specific add-on PEFT modules to adapt to new languages while freezing the base MASR. Empirical results across five low-resource languages show that Adapter-based XLA within PELE yields the strongest performance, outperforming several baselines and approaching the continual joint training upper bound with far fewer parameters. The findings highlight the importance of using module-based, adapter-style approaches over weight- or input-focused PEFTs for MASR language extension, and they demonstrate the practical potential for scalable multilingual extension.

Abstract

Covering all languages with a multilingual speech recognition model (MASR) is very difficult. Performing language extension on top of an existing MASR is a desirable choice. In this study, the MASR continual learning problem is probabilistically decomposed into language identity prediction (LP) and cross-lingual adaptation (XLA) sub-problems. Based on this, we propose an architecture-based framework for language extension that can fundamentally solve catastrophic forgetting, debudded as PELE. PELE is designed to be parameter-efficient, incrementally incorporating an add-on module to adapt to a new language. Specifically, different parameter-efficient fine-tuning (PEFT) modules and their variants are explored as potential candidates to perform XLA. Experiments are carried out on 5 new languages with a wide range of low-resourced data sizes. The best-performing PEFT candidate can achieve satisfactory performance across all languages and demonstrates superiority in three of five languages over the continual joint learning setting. Notably, PEFT methods focusing on weight parameters or input features are revealed to be limited in performance, showing significantly inferior extension capabilities compared to inserting a lightweight module in between layers such as an Adapter.
Paper Structure (16 sections, 3 equations, 1 figure, 1 table)

This paper contains 16 sections, 3 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. PELE
  3. CL-MASR Problem Decomposition
  4. CL Modification to MASR
  5. PELE Formulation
  6. Experimental Setup
  7. Base Model & Dataset
  8. MASR Continual Learning
  9. Baselines
  10. PELE
  11. Results and Analysis
  12. Results of the Decomposed Sub-problems
  13. Language Prediction (LP)
  14. Cross-lingual Adaptation (XLA)
  15. Comparison between PELE and Baselines
  16. ...and 1 more sections

Figures (1)

  • Figure 1: First row: The accuracy of different LID prediction methods. (a) on the original 10 languages; (b) on the new 5 languages. Second row: The CER results of different PEFT methods and baselines when performing XLA on the unseen language Japanese (ja). (c) the vocabulary layer is low-rank updated; (d) the vocabulary layer is fully updated.