Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Federated Learning of Large ASR Models in the Real World

Yonghui Xiao, Yuxin Ding, Changwan Ryu, Petr Zadrazil, Francoise Beaufays

TL;DR

This paper presents a systematic solution to train the full-size ASR models of 130M parameters with FL and is the first paper showing FL can improve the ASR model quality with a set of proposed methods to refine the quality of data and labels of clients.

Abstract

Federated learning (FL) has shown promising results on training machine learning models with privacy preservation. However, for large models with over 100 million parameters, the training resource requirement becomes an obstacle for FL because common devices do not have enough memory and computation power to finish the FL tasks. Although efficient training methods have been proposed, it is still a challenge to train the large models like Conformer based ASR. This paper presents a systematic solution to train the full-size ASR models of 130M parameters with FL. To our knowledge, this is the first real-world FL application of the Conformer model, which is also the largest model ever trained with FL so far. And this is the first paper showing FL can improve the ASR model quality with a set of proposed methods to refine the quality of data and labels of clients. We demonstrate both the training efficiency and the model quality improvement in real-world experiments.

Federated Learning of Large ASR Models in the Real World

TL;DR

This paper presents a systematic solution to train the full-size ASR models of 130M parameters with FL and is the first paper showing FL can improve the ASR model quality with a set of proposed methods to refine the quality of data and labels of clients.

Abstract

Federated learning (FL) has shown promising results on training machine learning models with privacy preservation. However, for large models with over 100 million parameters, the training resource requirement becomes an obstacle for FL because common devices do not have enough memory and computation power to finish the FL tasks. Although efficient training methods have been proposed, it is still a challenge to train the large models like Conformer based ASR. This paper presents a systematic solution to train the full-size ASR models of 130M parameters with FL. To our knowledge, this is the first real-world FL application of the Conformer model, which is also the largest model ever trained with FL so far. And this is the first paper showing FL can improve the ASR model quality with a set of proposed methods to refine the quality of data and labels of clients. We demonstrate both the training efficiency and the model quality improvement in real-world experiments.
Paper Structure (8 sections, 2 equations, 2 figures, 3 tables, 1 algorithm)

This paper contains 8 sections, 2 equations, 2 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Training Efficiency
  3. Model Quality
  4. Experimental Results
  5. Experiment Settings
  6. Training Efficiency
  7. Model Quality
  8. Conclusions

Figures (2)

  • Figure 1: The overview of the FL system. 1$\sim$5 are the FL steps in a round. There are 3 types of data on clients: the input audio data, the original transcript from the incumbent ASR and the final transcript based on user edits.
  • Figure 2: WER trade-off between general WER and target wER.