Hippocampus: An Efficient and Scalable Memory Module for Agentic AI

TL;DR

Hippocampus is introduced, an agentic memory management system that uses compact binary signatures for semantic search and lossless token-ID streams for exact content reconstruction that scales linearly with memory size, making it suitable for long-horizon agentic deployments.

Abstract

Agentic AI require persistent memory to store user-specific histories beyond the limited context window of LLMs. Existing memory systems use dense vector databases or knowledge-graph traversal (or hybrid), incurring high retrieval latency and poor storage scalability. We introduce Hippocampus, an agentic memory management system that uses compact binary signatures for semantic search and lossless token-ID streams for exact content reconstruction. Its core is a Dynamic Wavelet Matrix (DWM) that compresses and co-indexes both streams to support ultra-fast search in the compressed domain, thus avoiding costly dense-vector or graph computations. This design scales linearly with memory size, making it suitable for long-horizon agentic deployments. Empirically, our evaluation shows that Hippocampus reduces end-to-end retrieval latency by up to 31$\times$ and cuts per-query token footprint by up to 14$\times$, while maintaining accuracy on both LoCoMo and LongMemEval benchmarks.

Figures (12)

• Figure 1: An illustration of memory taxonomy of Agentic AI.
• Figure 2: Illustration of raw text, token-id, and word embedding.
• Figure 3: An analysis of SOTA agent memory systems across three critical metrics. Lower values are better for Avg. Token Consumption and Avg. Total Time, while higher is better for F1 Score. The plot illustrates that existing systems force a compromise, as none are able to simultaneously achieve high accuracy and high efficiency in the ideal design space indicated by the red star marker.
• Figure 4: Breakdown of the end-to-end retrieval latency for SOTA agentic AI memory modules.
• Figure 5: Illustration of the memory construction pipeline in Hippocampus. DWM denotes our proposed Dynamic Wavelet Matrix, while the subscript $(\cdot)_2$ indicates the binary representation of an integer, for example, token-id. The first row of the DWM serves as the entry-level index, marking the start and end positions of each token in the Content Serialization (e.g., StartIndex$\alpha$ and EndIndex$\beta$).