Words to Waves: Emotion-Adaptive Music Recommendation System
Apoorva Chavali, Reeve Menezes
TL;DR
The paper addresses the challenge of aligning music recommendations with a user’s real-time emotional state by mapping natural language inputs to the $V$-$A$ (Valence-Arousal) space using transformer-based embeddings. It introduces a Wide-and-Deep-inspired framework that fuses a VA predictor derived from a transformer with a memorization component capturing $emotion\times user$ and $emotion\times artist$ patterns, trained on EmoBank, Last.fm, and Spotify lyrics data. The approach achieves robust VA regression with rapid convergence (≈5 epochs) and a validation $R^2$ around $0.43$, demonstrating improved emotional relevance over baselines like larger pre-trained models. This emotion-adaptive scheme enables instant, emotionally consistent recommendations and broader artist exploration, while highlighting data limitations and the potential for richer, cross-dataset integration in future work.
Abstract
Current recommendation systems often tend to overlook emotional context and rely on historical listening patterns or static mood tags. This paper introduces a novel music recommendation framework employing a variant of Wide and Deep Learning architecture that takes in real-time emotional states inferred directly from natural language as inputs and recommends songs that closely portray the mood. The system captures emotional contexts from user-provided textual descriptions by using transformer-based embeddings, which were finetuned to predict the emotional dimensions of valence-arousal. The deep component of the architecture utilizes these embeddings to generalize unseen emotional patterns, while the wide component effectively memorizes user-emotion and emotion-genre associations through cross-product features. Experimental results show that personalized music selections positively influence the user's emotions and lead to a significant improvement in emotional relevance.