MemTrust: A Zero-Trust Architecture for Unified AI Memory System

TL;DR

MemTrust presents a hardware-backed zero-trust architecture for a unified AI memory system that preserves data sovereignty while enabling cross-application context sharing. It introduces a five-layer model (Storage, Extraction & Update, Learning & Evolution, Retrieval, Governance) implemented within TEEs, with multiple hardware backends and a standard Open Memory Protocol (UMP) to facilitate porting and interoperability. Key innovations include a Context from MemTrust protocol for cross-application sharing, side-channel hardened retrieval, cryptographic erasure, attestation-bound governance, and a dual-layer cognitive engine that separates episodic and semantic profile memory. The work demonstrates that secure, scalable memory across agents and tools is achievable with modest overhead (e.g., under $<20\%$ in evaluations) and offers a roadmap to ecosystem standardization via UMP and memory adapters, enabling a practical memory infrastructure for privacy-preserving AI collaboration.

Abstract

AI memory systems are evolving toward unified context layers that enable efficient cross-agent collaboration and multi-tool workflows, facilitating better accumulation of personal data and learning of user preferences. However, centralization creates a trust crisis where users must entrust cloud providers with sensitive digital memory data. We identify a core tension between personalization demands and data sovereignty: centralized memory systems enable efficient cross-agent collaboration but expose users' sensitive data to cloud provider risks, while private deployments provide security but limit collaboration. To resolve this tension, we aim to achieve local-equivalent security while enabling superior maintenance efficiency and collaborative capabilities. We propose a five-layer architecture abstracting common functional components of AI memory systems: Storage, Extraction, Learning, Retrieval, and Governance. By applying TEE protection to each layer, we establish a trustworthy framework. Based on this, we design MemTrust, a hardware-backed zero-trust architecture that provides cryptographic guarantees across all layers. Our contributions include the five-layer abstraction, "Context from MemTrust" protocol for cross-application sharing, side-channel hardened retrieval with obfuscated access patterns, and comprehensive security analysis. The architecture enables third-party developers to port existing systems with acceptable development costs, achieving system-wide trustworthiness. We believe that AI memory plays a crucial role in enhancing the efficiency and collaboration of agents and AI tools. AI memory will become the foundational infrastructure for AI agents, and MemTrust serves as a universal trusted framework for AI memory systems, with the goal of becoming the infrastructure of memory infrastructure.

Figures (5)

• Figure 1: Multi-Agent Collaboration workflow via TEE-protected Unified AI Memory. The User defines requirements with the Chat Agent (1), which securely persists the context within the TEE boundary (2). Later, the Coding Agent retrieves this shared context (4-5) to fulfill a generic request (3), returning personalized code (6) without redundant prompting, all while maintaining end-to-end confidentiality.
• Figure 2: MemTrust Five-Layer Architecture Overview
• Figure 3: TEE-Protected API Proxy Architecture with PII Masking
• Figure 4: VMPL0/1 Architecture
• Figure 5: Oblivious Expansion and Filtering Process for Side-Channel Resistant Retrieval