Intelligent Agents with Emotional Intelligence: Current Trends, Challenges, and Future Prospects

TL;DR

This survey provides a holistic, cross-modal examination of intelligent agents endowed with emotional intelligence, spanning emotion understanding, affective cognition, and emotional expression. It analyzes data, model, and system-level challenges across modalities (text, speech, and vision) and discusses state-of-the-art solutions, including data augmentation, multimodal fusion, cognitive architectures, and diffusion/LLM-based methods. The work highlights limitations such as data scarcity, interpretability, and contextual grounding, and outlines future directions emphasizing scalable datasets, explainable models, ethical considerations, and the integration of foundation models with cognitive frameworks. By offering a unified view of current trends and practical pathways, it aims to accelerate the development of naturalistic, empathetic agents suitable for diverse human–computer interaction applications.

Abstract

The development of agents with emotional intelligence is becoming increasingly vital due to their significant role in human-computer interaction and the growing integration of computer systems across various sectors of society. Affective computing aims to design intelligent systems that can recognize, evoke, and express human emotions, thereby emulating human emotional intelligence. While previous reviews have focused on specific aspects of this field, there has been limited comprehensive research that encompasses emotion understanding, elicitation, and expression, along with the related challenges. This survey addresses this gap by providing a holistic overview of core components of artificial emotion intelligence. It covers emotion understanding through multimodal data processing, as well as affective cognition, which includes cognitive appraisal, emotion mapping, and adaptive modulation in decision-making, learning, and reasoning. Additionally, it addresses the synthesis of emotional expression across text, speech, and facial modalities to enhance human-agent interaction. This paper identifies and analyzes the key challenges and issues encountered in the development of affective systems, covering state-of-the-art methodologies designed to address them. Finally, we highlight promising future directions, with particular emphasis on the potential of generative technologies to advance affective computing.

Figures (12)

• Figure 1: Overview of Intelligent Agent with Emotional Intelligence
• Figure 2: Distribution of the 298 included studies across the three core capabilities of emotional intelligence in intelligent agents.
• Figure 3: Breakdown of Emotion Understanding research by modality
• Figure 4: Breakdown of Emotional Expression Synthesis research by output modality.
• Figure 5: Categorization of Affective Cognition research by theoretical and computational focus.