Both Matter: Enhancing the Emotional Intelligence of Large Language Models without Compromising the General Intelligence

TL;DR

This work tackles the challenge of enhancing emotional intelligence in large language models without sacrificing general intelligence. It introduces EiBench, a comprehensive EI task suite spanning emotion perception cognition and expression, and MoEI, a modular, parameter-efficient framework that expands EI capacity via MoLoRA blocks and uses a router to balance EI and GI processing. Empirical results across Flan-T5 and LLaMA-2-Chat backbones show MoEI improves EI across tasks while preserving core GI capabilities and even aiding OOD robustness. Ablation studies and cross-task evaluations validate the contribution of modular expansion, intra-modulation, and inter-modulation to the observed gains. The work offers a practical path to deploy EI-enhanced LLMs in real-world assistants with maintained reasoning, knowledge, and reading comprehension, supported by open benchmarks and rigorous analyses.

Abstract

Emotional Intelligence (EI), consisting of emotion perception, emotion cognition and emotion expression, plays the critical roles in improving user interaction experience for the current large language model (LLM) based conversational general AI assistants. Previous works mainly focus on raising the emotion perception ability of them via naive fine-tuning on EI-related classification or regression tasks. However, this leads to the incomplete enhancement of EI and catastrophic forgetting of the general intelligence (GI). To this end, we first introduce \textsc{EiBench}, a large-scale collection of EI-related tasks in the text-to-text formation with task instructions that covers all three aspects of EI, which lays a solid foundation for the comprehensive EI enhancement of LLMs. Then a novel \underline{\textbf{Mo}}dular \underline{\textbf{E}}motional \underline{\textbf{I}}ntelligence enhancement method (\textbf{MoEI}), consisting of Modular Parameter Expansion and intra-inter modulation, is proposed to comprehensively enhance the EI of LLMs without compromise their GI. Extensive experiments on two representative LLM-based assistants, Flan-T5 and LLaMA-2-Chat, demonstrate the effectiveness of MoEI to improving EI while maintain GI.

Figures (9)

• Figure 1: Comparison of EI and GI before (in orange) and after (in blue) the EI-enhancement via naive fine-tuning based on the Flan-T5-XL (3B) backbone.
• Figure 2: Overview of EiBench. 15 EI-related tasks are categorized into 3 main categories: emotion perception, emotion cognition, and emotion expression.
• Figure 3: The overall architecture of our proposed MoEI framework, which consists of two techniques, modular parameter expansion and intra-inter modulation. Red and blue lines represent the forward flow of the EI- and GI related inputs that participate in the Intra- and Inter-Modulation, respectively.
• Figure 4: Results of EI and GI of different methods on the larger Flan-T5-XXL (11B) backbone.
• Figure 5: Results of the EI-enhancement on the EQ-Bench. The LLM backbones are Flan-T5-XXL (11B) and LLaMA-2-Chat-13B.