Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Cross-Lingual Meta-Learning Method Based on Domain Adaptation for Speech Emotion Recognition

David-Gabriel Ion, Răzvan-Alexandru Smădu, Dumitru-Clementin Cercel, Florin Pop, Mihaela-Claudia Cercel

TL;DR

This work explores the model's performance in limited data, specifically for speech emotion recognition, and proposes a series of improvements over the multistage meta-learning method, which incorporates a large pre-trained backbone and a prototypical network, making the methods more feasible and applicable.

Abstract

Best-performing speech models are trained on large amounts of data in the language they are meant to work for. However, most languages have sparse data, making training models challenging. This shortage of data is even more prevalent in speech emotion recognition. Our work explores the model's performance in limited data, specifically for speech emotion recognition. Meta-learning specializes in improving the few-shot learning. As a result, we employ meta-learning techniques on speech emotion recognition tasks, accent recognition, and person identification. To this end, we propose a series of improvements over the multistage meta-learning method. Unlike other works focusing on smaller models due to the high computational cost of meta-learning algorithms, we take a more practical approach. We incorporate a large pre-trained backbone and a prototypical network, making our methods more feasible and applicable. Our most notable contribution is an improved fine-tuning technique during meta-testing that significantly boosts the performance on out-of-distribution datasets. This result, together with incremental improvements from several other works, helped us achieve accuracy scores of 83.78% and 56.30% for Greek and Romanian speech emotion recognition datasets not included in the training or validation splits in the context of 4-way 5-shot learning.

A Cross-Lingual Meta-Learning Method Based on Domain Adaptation for Speech Emotion Recognition

TL;DR

This work explores the model's performance in limited data, specifically for speech emotion recognition, and proposes a series of improvements over the multistage meta-learning method, which incorporates a large pre-trained backbone and a prototypical network, making the methods more feasible and applicable.

Abstract

Best-performing speech models are trained on large amounts of data in the language they are meant to work for. However, most languages have sparse data, making training models challenging. This shortage of data is even more prevalent in speech emotion recognition. Our work explores the model's performance in limited data, specifically for speech emotion recognition. Meta-learning specializes in improving the few-shot learning. As a result, we employ meta-learning techniques on speech emotion recognition tasks, accent recognition, and person identification. To this end, we propose a series of improvements over the multistage meta-learning method. Unlike other works focusing on smaller models due to the high computational cost of meta-learning algorithms, we take a more practical approach. We incorporate a large pre-trained backbone and a prototypical network, making our methods more feasible and applicable. Our most notable contribution is an improved fine-tuning technique during meta-testing that significantly boosts the performance on out-of-distribution datasets. This result, together with incremental improvements from several other works, helped us achieve accuracy scores of 83.78% and 56.30% for Greek and Romanian speech emotion recognition datasets not included in the training or validation splits in the context of 4-way 5-shot learning.
Paper Structure (25 sections, 2 equations, 1 figure, 10 tables)

This paper contains 25 sections, 2 equations, 1 figure, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Architecture Overview
  5. Feature Extractors
  6. Prototypical Network
  7. Training Procedure
  8. Meta-Learning.
  9. Fine-Tuning.
  10. Domain Adversarial Training.
  11. Experiments
  12. Datasets
  13. Person Identification
  14. Accent Recognition
  15. Speech Emotion Recognition
  16. ...and 10 more sections

Figures (1)

  • Figure 1: The model architecture consists of the Wav2Vec2 XLS-R 300M backbone, the prototypical network on the top branch, and the domain adversarial dataset discriminator on the bottom branch. $\theta_m$, $\theta_f$, and $\theta_d$ represent the parameters for the embedding model, feature extractor, and dataset discriminator, respectively, and the dotted lines represent the gradient flows. $\lambda$ controls the domain adaptation influence.