EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Models

TL;DR

Reflective Reinforcement Learning for Emotional Reasoning (EMO-R3) is proposed, a framework designed to enhance the emotional reasoning ability ofMultimodal Large Language Models and introduces Structured Emotional Thinking to guide the model to perform step-by-step emotional reasoning in a structured and interpretable manner.

Abstract

Multimodal Large Language Models (MLLMs) have shown remarkable progress in visual reasoning and understanding tasks but still struggle to capture the complexity and subjectivity of human emotions. Existing approaches based on supervised fine-tuning often suffer from limited generalization and poor interpretability, while reinforcement learning methods such as Group Relative Policy Optimization fail to align with the intrinsic characteristics of emotional cognition. To address these challenges, we propose Reflective Reinforcement Learning for Emotional Reasoning (EMO-R3), a framework designed to enhance the emotional reasoning ability of MLLMs. Specifically, we introduce Structured Emotional Thinking to guide the model to perform step-by-step emotional reasoning in a structured and interpretable manner, and design a Reflective Emotional Reward that enables the model to re-evaluate its reasoning based on visual-text consistency and emotional coherence. Extensive experiments demonstrate that EMO-R3 significantly improves both the interpretability and emotional intelligence of MLLMs, achieving superior performance across multiple visual emotional understanding benchmarks.

Figures (5)

• Figure 1: Illustration of the motivation.(a) SFT relies on human annotations but is constrained by fixed labels and limited categories, resulting in poor generalization and interpretability. It performs well on in-domain pairs like “landscape–awe” but struggles with out-of-domain or unseen cases (e.g., “movement-surprise”). (b) Although GRPO improves generalization, its think process is not emotion-oriented and weakly connected to the final answer (e.g., rethinking the last rollout yields “amusement”, while the prediction is “fear”).
• Figure 2: Architecture illustration of EMO-R3. The upper part presents the Structured Emotional Thinking prompt, which consists of three consecutive thinking steps followed by a final answer. The lower part illustrates the Reflective Emotional Reward mechanism, where multiple rollout samples are evaluated based on image–text consistency and emotional coherence, and are jointly optimized with the original Format and Accuracy rewards under the GRPO framework.
• Figure 3: Training and testing accuracy during the training process. DAPO fails to conduct complete training. A more detailed analysis of this failure is provided in \ref{['subsec:comp_exp']}.
• Figure 4: Case study between GRPO and EMO-R3 on the EmoSet dataset. Please see \ref{['subsec:casestudy']} for details.
• Figure 5: Efficiency analysis on the training process. See \ref{['subsec:time']}.