SimpleMem: Efficient Lifelong Memory for LLM Agents

TL;DR

This work tackles the challenge of long-term memory for LLM agents operating under fixed context windows. It introduces SimpleMem, a memory framework built on semantic lossless compression with three stages: Semantic Structured Compression, Recursive Memory Consolidation, and Adaptive Query-Aware Retrieval. Empirical results on the LoCoMo benchmark show substantial gains in F1 (e.g., 26.4% improvement over strong baselines) and dramatic reductions in inference tokens (up to 30×) across diverse backbone models, including both high-capacity and smaller open-source systems. The approach sustains high reasoning accuracy while significantly improving retrieval efficiency, enabling reliable lifelong interaction in complex environments.

Abstract

To support reliable long-term interaction in complex environments, LLM agents require memory systems that efficiently manage historical experiences. Existing approaches either retain full interaction histories via passive context extension, leading to substantial redundancy, or rely on iterative reasoning to filter noise, incurring high token costs. To address this challenge, we introduce SimpleMem, an efficient memory framework based on semantic lossless compression. We propose a three-stage pipeline designed to maximize information density and token utilization: (1) \textit{Semantic Structured Compression}, which applies entropy-aware filtering to distill unstructured interactions into compact, multi-view indexed memory units; (2) \textit{Recursive Memory Consolidation}, an asynchronous process that integrates related units into higher-level abstract representations to reduce redundancy; and (3) \textit{Adaptive Query-Aware Retrieval}, which dynamically adjusts retrieval scope based on query complexity to construct precise context efficiently. Experiments on benchmark datasets show that our method consistently outperforms baseline approaches in accuracy, retrieval efficiency, and inference cost, achieving an average F1 improvement of 26.4% while reducing inference-time token consumption by up to 30-fold, demonstrating a superior balance between performance and efficiency. Code is available at https://github.com/aiming-lab/SimpleMem.

Figures (3)

• Figure 1: Performance vs. Efficiency Trade-off. Comparison of Average F1 against Average Token Cost on the LoCoMo benchmark. SimpleMem occupies the ideal top-left position, achieving high accurac with minimal token consumption ($\sim$550 tokens).
• Figure 2: The SimpleMem Architecture. SimpleMem mitigates context inflation through three stages. (1) Semantic Structured Compression filters redundant interaction content and reformulates raw dialogue into compact, context-independent memory units. (2) Recursive Consolidation incrementally organizes related memory units into higher-level abstract representations, reducing redundancy in long-term memory. (3) Adaptive Query-Aware Retrieval dynamically adjusts retrieval scope based on query complexity, enabling efficient context construction under constrained token budgets.
• Figure 3: A Case of SimpleMem for Long-Term Multi-Session Dialogues. SimpleMem processes multi-session dialogues by filtering redundant content, normalizing temporal references, and organizing memories into compact representations. During retrieval, it adaptively combines semantic, lexical, and symbolic signals to select relevant entries.