MemEmo: Evaluating Emotion in Memory Systems of Agents

TL;DR

An emotion-enhanced memory evaluation benchmark is proposed to assess the performance of mainstream and state-of-the-art memory systems in handling affective information and suggest a new trajectory for future research and system optimization.

Abstract

Memory systems address the challenge of context loss in Large Language Model during prolonged interactions. However, compared to human cognition, the efficacy of these systems in processing emotion-related information remains inconclusive. To address this gap, we propose an emotion-enhanced memory evaluation benchmark to assess the performance of mainstream and state-of-the-art memory systems in handling affective information. We developed the \textbf{H}uman-\textbf{L}ike \textbf{M}emory \textbf{E}motion (\textbf{HLME}) dataset, which evaluates memory systems across three dimensions: emotional information extraction, emotional memory updating, and emotional memory question answering. Experimental results indicate that none of the evaluated systems achieve robust performance across all three tasks. Our findings provide an objective perspective on the current deficiencies of memory systems in processing emotional memories and suggest a new trajectory for future research and system optimization.

Figures (3)

• Figure 1: An example illustrates the lack of emotion processing by the memory system in HCI dialogues.
• Figure 2: The HLME dataset construction process pipeline.
• Figure 3: HLME dataset evaluation scheme.