Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Aeon: High-Performance Neuro-Symbolic Memory Management for Long-Horizon LLM Agents

Mustafa Arslan

TL;DR

Aeon reframes memory in LLM-driven agents as a persistent, OS-like resource rather than a passive bag of embeddings, addressing the $O(N^2)$ self-attention bottleneck and the Lost in the Middle phenomenon. It introduces a Core-Shell architecture with a high-performance Atlas spatial index, a Trace episodic graph, and a Semantic Lookaside Buffer (SLB) to achieve sub-millisecond latency in long-horizon reasoning. The key contributions are a memory-mapped A-atlas, a neuro-symbolic Trace DAG, and a zero-copy C++/Python interface, yielding $<1\text{ ms}$ retrieval on conversational workloads and scalable performance up to $N=10^6$ nodes with logarithmic growth. Practically, Aeon enables persistent, structured memory for autonomous agents, combining speed, coherence, and auditability in a unified kernel-level memory system.

Abstract

Large Language Models (LLMs) are fundamentally constrained by the quadratic computational cost of self-attention and the "Lost in the Middle" phenomenon, where reasoning capabilities degrade as context windows expand. Existing solutions, primarily "Flat RAG" architectures relying on vector databases, treat memory as an unstructured bag of embeddings. This approach fails to capture the hierarchical and temporal structure of long-horizon interactions, leading to "Vector Haze", the retrieval of disjointed facts lacking episodic continuity. We propose Aeon, a Neuro-Symbolic Cognitive Operating System that redefines memory not as a static store, but as a managed OS resource. Aeon structures memory into a Memory Palace (a spatial index implemented via Atlas, a SIMD-accelerated Page-Clustered Vector Index that combines small-world graph navigation with B+ Tree-style disk locality to minimize read amplification) and a Trace (a neuro-symbolic episodic graph). We introduce the Semantic Lookaside Buffer (SLB), a predictive caching mechanism that exploits conversational locality to achieve sub-millisecond retrieval latencies. Benchmarks demonstrate that Aeon achieves < 1ms retrieval latency on conversational workloads while ensuring state consistency via a zero-copy C++/Python bridge, effectively enabling persistent, structured memory for autonomous agents.

Aeon: High-Performance Neuro-Symbolic Memory Management for Long-Horizon LLM Agents

TL;DR

Aeon reframes memory in LLM-driven agents as a persistent, OS-like resource rather than a passive bag of embeddings, addressing the self-attention bottleneck and the Lost in the Middle phenomenon. It introduces a Core-Shell architecture with a high-performance Atlas spatial index, a Trace episodic graph, and a Semantic Lookaside Buffer (SLB) to achieve sub-millisecond latency in long-horizon reasoning. The key contributions are a memory-mapped A-atlas, a neuro-symbolic Trace DAG, and a zero-copy C++/Python interface, yielding retrieval on conversational workloads and scalable performance up to nodes with logarithmic growth. Practically, Aeon enables persistent, structured memory for autonomous agents, combining speed, coherence, and auditability in a unified kernel-level memory system.

Abstract

Large Language Models (LLMs) are fundamentally constrained by the quadratic computational cost of self-attention and the "Lost in the Middle" phenomenon, where reasoning capabilities degrade as context windows expand. Existing solutions, primarily "Flat RAG" architectures relying on vector databases, treat memory as an unstructured bag of embeddings. This approach fails to capture the hierarchical and temporal structure of long-horizon interactions, leading to "Vector Haze", the retrieval of disjointed facts lacking episodic continuity. We propose Aeon, a Neuro-Symbolic Cognitive Operating System that redefines memory not as a static store, but as a managed OS resource. Aeon structures memory into a Memory Palace (a spatial index implemented via Atlas, a SIMD-accelerated Page-Clustered Vector Index that combines small-world graph navigation with B+ Tree-style disk locality to minimize read amplification) and a Trace (a neuro-symbolic episodic graph). We introduce the Semantic Lookaside Buffer (SLB), a predictive caching mechanism that exploits conversational locality to achieve sub-millisecond retrieval latencies. Benchmarks demonstrate that Aeon achieves < 1ms retrieval latency on conversational workloads while ensuring state consistency via a zero-copy C++/Python bridge, effectively enabling persistent, structured memory for autonomous agents.
Paper Structure (40 sections, 5 equations, 3 figures, 2 algorithms)

This paper contains 40 sections, 5 equations, 3 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Retrieval-Augmented Generation (RAG)
  4. Memory-Augmented LLMs
  5. Neuro-Symbolic Knowledge Graphs
  6. Operating System Primitives for AI
  7. System Architecture
  8. Design Philosophy: The Core-Shell Model
  9. The Atlas: Spatial Memory Kernel
  10. Data Structure
  11. Storage and Access
  12. Greedy SIMD Descent
  13. The Trace: Episodic Context Graph
  14. Node Types
  15. Edge Structure
  16. ...and 25 more sections

Figures (3)

  • Figure 1: Math kernel throughput across SIMD implementations. AVX-512 achieves 20$\times$ speedup over scalar C++ and nearly 2000$\times$ over interpreted Python. Error bars indicate 25th/75th percentiles over 5 runs.
  • Figure 2: Cumulative Distribution Function of query latency. Aeon (Warm) exhibits a bimodal distribution: 85% of queries complete in under 0.1ms (SLB hits), while the remaining 15% form a "long tail" up to 2.5ms (SLB misses). HNSW shows constant latency around 1.5ms.
  • Figure 3: Query latency as a function of database size. Flat search scales linearly (dashed), while Aeon Atlas scales logarithmically (solid). At 1 million nodes, Aeon is 40$\times$ faster than brute-force search.