EEmo-Logic: A Unified Dataset and Multi-Stage Framework for Comprehensive Image-Evoked Emotion Assessment

TL;DR

EEmo-Logic tackles the multi-dimensional and subjective nature of image-evoked emotions by building EEmoDB, the largest instruction-based dataset for AICA, and a two-stage MLLM framework that combines LoRA-based supervised fine-tuning with group relative preference optimization. The dataset integrates CES and DES across 1.2M QA instructions and a 36k fine-grained assessment set, enabling robust perception, reasoning, ranking, and VAD/DEC evaluation. The approach demonstrates state-of-the-art performance in in-domain benchmarks and strong zero-shot generalization to cross-domain emotion datasets, highlighting the value of unified CES-DES reasoning and task-specific GRPO rewards for computational empathy. The work has practical implications for empathetic HCI and AI systems, while acknowledging the need to manage subjectivity and biases inherent in emotion understanding.

Abstract

Understanding the multi-dimensional attributes and intensity nuances of image-evoked emotions is pivotal for advancing machine empathy and empowering diverse human-computer interaction applications. However, existing models are still limited to coarse-grained emotion perception or deficient reasoning capabilities. To bridge this gap, we introduce EEmoDB, the largest image-evoked emotion understanding dataset to date. It features $5$ analysis dimensions spanning $5$ distinct task categories, facilitating comprehensive interpretation. Specifically, we compile $1.2M$ question-answering (QA) pairs (EEmoDB-QA) from $125k$ images via automated generation, alongside a $36k$ dataset (EEmoDB-Assess) curated from $25k$ images for fine-grained assessment. Furthermore, we propose EEmo-Logic, an all-in-one multimodal large language model (MLLM) developed via instruction fine-tuning and task-customized group relative preference optimization (GRPO) with novel reward design. Extensive experiments demonstrate that EEmo-Logic achieves robust performance in in-domain and cross-domain datasets, excelling in emotion QA and fine-grained assessment. The code is available at https://anonymous.4open.science/r/EEmoLogic.

Figures (13)

• Figure 1: Trained on our constructed EEmoDB via a two-stage paradigm, EEmo-Logic (a) achieves robust question-answering performance on EEmo-Bench, and (b) demonstrates strong capabilities in fine-grained emotion assessment and causal reasoning.
• Figure 2: EEmoDB construction pipeline. (a) Selects source datasets based on $5$ dimensions. (b) Refines emotional labels via $3$ distinct processes. (c) Compiles EEmoDB-QA subset for perception, ranking, and description tasks using rule-based and model-assisted methods. (d) Curates EEmoDB-Assess subset to support GRPO training for fine-grained ranking, VAD scoring, and DEC classification.
• Figure 3: Comparative distributions across the dataset. (a)-(c) show the score distributions for valence, arousal, and dominance, respectively. (d) illustrates the overall emotional frequency statistics in the EEmoDB-QA subset.
• Figure 4: The EEmo-Logic training framework. The pipeline comprises two stages: 1) Supervised LoRA SFT on QA data for foundational QA capability; and 2) GRPO training on curated emotion labels to refine emotion assessment and elicit reasoning.
• Figure 5: Visualization of EEmo-Logic’s fine-grained emotion assessment and reasoning capabilities in $3$ tasks.