Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leveraging LLM Embeddings for Cross Dataset Label Alignment and Zero Shot Music Emotion Prediction

Renhang Liu, Abhinaba Roy, Dorien Herremans

TL;DR

This work compute LLM embeddings for emotion labels and apply non-parametric clustering to group similar labels, across multiple datasets containing disjoint labels, and introduces an alignment regularization that enables dissociation of MERT embeddings from different clusters.

Abstract

In this work, we present a novel method for music emotion recognition that leverages Large Language Model (LLM) embeddings for label alignment across multiple datasets and zero-shot prediction on novel categories. First, we compute LLM embeddings for emotion labels and apply non-parametric clustering to group similar labels, across multiple datasets containing disjoint labels. We use these cluster centers to map music features (MERT) to the LLM embedding space. To further enhance the model, we introduce an alignment regularization that enables dissociation of MERT embeddings from different clusters. This further enhances the model's ability to better adaptation to unseen datasets. We demonstrate the effectiveness of our approach by performing zero-shot inference on a new dataset, showcasing its ability to generalize to unseen labels without additional training.

Leveraging LLM Embeddings for Cross Dataset Label Alignment and Zero Shot Music Emotion Prediction

TL;DR

This work compute LLM embeddings for emotion labels and apply non-parametric clustering to group similar labels, across multiple datasets containing disjoint labels, and introduces an alignment regularization that enables dissociation of MERT embeddings from different clusters.

Abstract

In this work, we present a novel method for music emotion recognition that leverages Large Language Model (LLM) embeddings for label alignment across multiple datasets and zero-shot prediction on novel categories. First, we compute LLM embeddings for emotion labels and apply non-parametric clustering to group similar labels, across multiple datasets containing disjoint labels. We use these cluster centers to map music features (MERT) to the LLM embedding space. To further enhance the model, we introduce an alignment regularization that enables dissociation of MERT embeddings from different clusters. This further enhances the model's ability to better adaptation to unseen datasets. We demonstrate the effectiveness of our approach by performing zero-shot inference on a new dataset, showcasing its ability to generalize to unseen labels without additional training.

Paper Structure

This paper contains 18 sections, 10 equations, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Present Work
  4. Emotion Label Embedding
  5. Clustering of Emotion Embeddings
  6. Mapping MERT Features to the Label Embedding Space
  7. Input Projection
  8. Self-Attention
  9. Output Projection
  10. Alignment Loss
  11. Alignment Regularization
  12. Zero Shot Classification
  13. Experiments and Results
  14. Datasets
  15. Experimental Setup
  16. ...and 3 more sections

Figures (2)

  • Figure 1: Overview of our approach.
  • Figure 2: Graph of labels taken from CAL500 and Emotify. Nodes with same color belong to the same cluster. Labels (nodes) from the same cluster are shown connected by an edge. Spatial distance here is irrelevent.