Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Semi-Supervised Self-Learning Enhanced Music Emotion Recognition

Yifu Sun, Xulong Zhang, Monan Zhou, Wei Li

TL;DR

This paper tackles music emotion recognition under limited labeled data by leveraging segment-based augmentation and a label-noise robust training pipeline. It introduces a semi-supervised self-learning (SSSL) framework that partitions expanded segment data into clean and noisy sets using a two-component Gaussian Mixture Model, then applies pseudo-label sharpening, mixup augmentation, and KL-based consistency to mitigate confirmation bias. A two-stage approach first learns a robust segment-level emotion predictor and then aggregates segment probabilities into song-level features for final emotion classification with a downstream classifier. Experiments across PMEmo, EiM, and 4Q datasets show that the proposed method achieves better or competitive performance relative to baselines, demonstrating the effectiveness of handling noisy labels and exploiting segment-level dynamics for MER.

Abstract

Music emotion recognition (MER) aims to identify the emotions conveyed in a given musical piece. However, currently, in the field of MER, the available public datasets have limited sample sizes. Recently, segment-based methods for emotion-related tasks have been proposed, which train backbone networks on shorter segments instead of entire audio clips, thereby naturally augmenting training samples without requiring additional resources. Then, the predicted segment-level results are aggregated to obtain the entire song prediction. The most commonly used method is that the segment inherits the label of the clip containing it, but music emotion is not constant during the whole clip. Doing so will introduce label noise and make the training easy to overfit. To handle the noisy label issue, we propose a semi-supervised self-learning (SSSL) method, which can differentiate between samples with correct and incorrect labels in a self-learning manner, thus effectively utilizing the augmented segment-level data. Experiments on three public emotional datasets demonstrate that the proposed method can achieve better or comparable performance.

Semi-Supervised Self-Learning Enhanced Music Emotion Recognition

TL;DR

This paper tackles music emotion recognition under limited labeled data by leveraging segment-based augmentation and a label-noise robust training pipeline. It introduces a semi-supervised self-learning (SSSL) framework that partitions expanded segment data into clean and noisy sets using a two-component Gaussian Mixture Model, then applies pseudo-label sharpening, mixup augmentation, and KL-based consistency to mitigate confirmation bias. A two-stage approach first learns a robust segment-level emotion predictor and then aggregates segment probabilities into song-level features for final emotion classification with a downstream classifier. Experiments across PMEmo, EiM, and 4Q datasets show that the proposed method achieves better or competitive performance relative to baselines, demonstrating the effectiveness of handling noisy labels and exploiting segment-level dynamics for MER.

Abstract

Music emotion recognition (MER) aims to identify the emotions conveyed in a given musical piece. However, currently, in the field of MER, the available public datasets have limited sample sizes. Recently, segment-based methods for emotion-related tasks have been proposed, which train backbone networks on shorter segments instead of entire audio clips, thereby naturally augmenting training samples without requiring additional resources. Then, the predicted segment-level results are aggregated to obtain the entire song prediction. The most commonly used method is that the segment inherits the label of the clip containing it, but music emotion is not constant during the whole clip. Doing so will introduce label noise and make the training easy to overfit. To handle the noisy label issue, we propose a semi-supervised self-learning (SSSL) method, which can differentiate between samples with correct and incorrect labels in a self-learning manner, thus effectively utilizing the augmented segment-level data. Experiments on three public emotional datasets demonstrate that the proposed method can achieve better or comparable performance.

Paper Structure

This paper contains 17 sections, 10 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Method
  3. SSSL Framework
  4. Task Formulation
  5. Training Samples Partition
  6. Modified Semi-supervised Learning
  7. Loss Function
  8. Song-level Decisions
  9. Experiments
  10. Experiment Setup
  11. Datasets
  12. Audio Pre-processing
  13. Results
  14. Performance at Different Segment Duration
  15. Validation Experiment
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Flow chart of the proposed method: blue arrows indicate the clean set (Gaussian distribution with smaller mean), red arrows indicate the noisy set, which is treated as the unlabeled set in subsequent semi-supervised learning where the clean set serves as the labeled set
  • Figure 2: Segment-level classifier output on test sample MT0010465830