GlobalMood: A cross-cultural benchmark for music emotion recognition

TL;DR

GlobalMood tackles the bias in music emotion recognition datasets by creating a cross-cultural benchmark with 1,180 songs from 59 countries and 2,519 participants. Using a bottom-up tagging and large-scale rating workflow, it captures culturally specific emotion descriptors and reveals a shared valence–arousal structure alongside cross-cultural divergences. The study demonstrates that fine-tuning multimodal models on GlobalMood substantially improves performance in non-English contexts, underscoring the importance of culturally representative data for cross-lingual MIR and NLP tasks. Overall, GlobalMood provides a scalable framework and dataset to advance culturally aware emotion understanding in music and multimodal systems.

Abstract

Human annotations of mood in music are essential for music generation and recommender systems. However, existing datasets predominantly focus on Western songs with terms derived from English, which may limit generalizability across diverse linguistic and cultural backgrounds. We introduce 'GlobalMood', a novel cross-cultural benchmark dataset comprising 1,180 songs sampled from 59 countries, with large-scale annotations collected from 2,519 individuals across five culturally and linguistically distinct locations: U.S., France, Mexico, S. Korea, and Egypt. Rather than imposing predefined emotion and mood categories, we implement a bottom-up, participant-driven approach to organically elicit culturally specific music-related emotion terms. We then recruit another pool of human participants to collect 988,925 ratings for these culture-specific descriptors. Our analysis confirms the presence of a valence-arousal structure shared across cultures, yet also reveals significant divergences in how certain emotion terms (despite being dictionary equivalents) are perceived cross-culturally. State-of-the-art multimodal models benefit substantially from fine-tuning on our cross-culturally balanced dataset, particularly in non-English contexts. Broadly, our findings inform the ongoing debate on the universality versus cultural specificity of emotional descriptors, and our methodology can contribute to other multimodal and cross-lingual research.

Figures (4)

• Figure 1: Elicitation and refinement of music emotion terms through iterative participant chains. (A) Schematic illustration of the collaborative tagging process within a participant chain. Participants contribute new emotion-related word tags for each song, rate the relevance of existing tags, and can also flag irrelevant content, creating a dynamic refinement system. (B) Twenty most reliable emotion tags in each language, ranked by their tag scores. Y-axis labels display tags in their original language (left) and English translations (right).
• Figure 2: Association between emotion terms across languages. (A) MDS visualization of the emotion terms based on mean ratings across the full song set. Terms positioned closer together exhibit similar rating patterns across songs, suggesting similar interpretations across languages. (B) Comparison of terms with direct translation equivalents across languages. The area size indicates the degree of semantic divergence despite apparent translation equivalence.
• Figure 3: Correlations between human ratings and multimodal model predictions. (A) Gemini models with zero-shot prompting showing increase in performance with newer models. (B) CLAP models in zero-shot and fine-tuned scenarios showing how the use of multilingual annotations can substantially increase performance. Gray dashed lines represent split-half reliability of human ratings using the Spearman-Brown formula as baseline reference of correlations achieved between humans. Error bars indicate 95% CI of mean correlation across songs.
• Figure :