AffectGPT-R1: Leveraging Reinforcement Learning for Open-Vocabulary Multimodal Emotion Recognition

TL;DR

AffectGPT-R1 introduces a reinforcement-learning framework for open-vocabulary multimodal emotion recognition by casting emotion-wheel–based metrics as rewards and optimizing with Group Relative Policy Optimization. It deploys a two-phase training regime (cold-start with descriptive data, then RL with open-vocabulary labels), five reward signals (plus length penalties) to balance thinking and answer quality, and a mechanism to prevent reward hacking. Empirical results show substantial gains on OV-MER and state-of-the-art performance on MER-UniBench, with insights on reward design, output mode, and data requirements. The work demonstrates the potential of RL to align model behavior with nuanced emotion understanding in multimodal contexts, and it provides resources to foster reproducibility and future exploration.

Abstract

Open-Vocabulary Multimodal Emotion Recognition (OV-MER) aims to predict emotions without being constrained by predefined label spaces, enabling fine-grained emotion understanding. Unlike traditional discriminative methods, OV-MER leverages generative models, such as large language models, to capture the full spectrum of emotions and employs emotion wheels (EWs) for metric calculation. Previous approaches (e.g., AffectGPT) primarily rely on token-level loss during training. However, this objective is misaligned with the metrics used in OV-MER, while these metrics cannot be optimized via gradient backpropagation. In this paper, we propose AffectGPT-R1, a reinforcement learning framework that formulates EW-based metrics as a reward function and employs a policy-based optimization strategy to maximize this reward. Additionally, we introduce an extra reasoning process and investigate its necessity in OV-MER. To further refine model behavior, we incorporate auxiliary rewards that constrain both reasoning and emotion prediction. To prevent reward hacking, we propose to incorporate length penalties during training. Experimental results show that AffectGPT-R1 achieves substantial improvements on OV-MER. Beyond this task, our approach also enhances generalized emotion understanding, attaining state-of-the-art performance on MER-UniBench. To the best of our knowledge, this is the first work to adapt the R1-style methodology for emotion understanding, revealing the impact of reasoning processes and reinforcement learning in this domain. Our code is provided in the supplementary material and will be released to facilitate future research.

Figures (3)

• Figure 1: Overall pipeline of AffectGPT-R1. Our training pipeline consists of two phases: cold start training and reinforcement learning. In the first phase, we adopt the framework of AffectGPT but replace the generated content, including both thinking and answers. In the second phase, we propose five rewards and introduce length penalties to mitigate reward hacking, followed by GRPO for policy updates.
• Figure 2: Left: Impact of $G$. "--" denotes the baseline model without reinforcement learning. Mid: Impact of data amount in cold-start training. This figure presents the performance of models trained with varying amounts of cold-start data and analyzes their influence on subsequent reinforcement learning. Right: Impact of data amount in reinforcement learning. "--" denotes the baseline model without reinforcement learning. This figure shows that using a larger amount of data in reinforcement learning leads to better performance.
• Figure 3: Emotion wheels. This paper uses five emotion wheels to calculate EW-based metrics.