CAIM: Development and Evaluation of a Cognitive AI Memory Framework for Long-Term Interaction with Intelligent Agents

TL;DR

CAIM addresses the challenge of long-term interactions with LLMs by introducing a cognitive AI-inspired memory framework that combines an ontology-based tagging scheme with a three-module architecture: Memory Controller, Memory Retrieval, and Post-Thinking. The framework differentiates between short-term and long-term memory, uses a Decision Unit to control memory usage, and employs context- and time-based relevance filtering to retrieve only pertinent memories for response generation. Experimental evaluation on the Generated Virtual Dataset across three LLMs shows that CAIM improves retrieval accuracy, response correctness, and contextual coherence, and enables memory storage across sessions, outperforming MemoryBank and TiM in key metrics. The work also discusses limitations, notably handling detailed queries and relative time references, and outlines future directions such as balancing memory detail with efficiency, enabling learning from queries, and validating CAIM in real-world scenarios.

Abstract

Large language models (LLMs) have advanced the field of artificial intelligence (AI) and are a powerful enabler for interactive systems. However, they still face challenges in long-term interactions that require adaptation towards the user as well as contextual knowledge and understanding of the ever-changing environment. To overcome these challenges, holistic memory modeling is required to efficiently retrieve and store relevant information across interaction sessions for suitable responses. Cognitive AI, which aims to simulate the human thought process in a computerized model, highlights interesting aspects, such as thoughts, memory mechanisms, and decision-making, that can contribute towards improved memory modeling for LLMs. Inspired by these cognitive AI principles, we propose our memory framework CAIM. CAIM consists of three modules: 1.) The Memory Controller as the central decision unit; 2.) the Memory Retrieval, which filters relevant data for interaction upon request; and 3.) the Post-Thinking, which maintains the memory storage. We compare CAIM against existing approaches, focusing on metrics such as retrieval accuracy, response correctness, contextual coherence, and memory storage. The results demonstrate that CAIM outperforms baseline frameworks across different metrics, highlighting its context-awareness and potential to improve long-term human-AI interactions.

Figures (2)

• Figure 1: Example of CAIM's ontology structure
• Figure 2: Workflow of CAIM - Illustration of the three modules, with colors highlighting the different possible paths within the system.