AI-native Memory 2.0: Second Me

TL;DR

The paper presents Second Me, an AI-native memory offload system that leverages memory parameterization across a three-layer architecture (L0-L2) to reduce cognitive load and enable context-aware memory management. It introduces an automated, privacy-preserving training pipeline combining SFT and DPO, and a CoT-oriented data-synthesis framework to adapt L2 models for Memory QA, Context Enhancement, and Context Critic tasks. Through automated evaluations and human case studies, the authors demonstrate performance gains from strong CoT configurations and DPO, while highlighting evaluation biases and the need for multimodal data. The work positions Second Me as a context-provider and orchestrator within a multi-agent ecosystem, and provides open-source tooling for local deployment and further development.

Abstract

Human interaction with the external world fundamentally involves the exchange of personal memory, whether with other individuals, websites, applications, or, in the future, AI agents. A significant portion of this interaction is redundant, requiring users to repeatedly provide the same information across different contexts. Existing solutions, such as browser-stored credentials, autofill mechanisms, and unified authentication systems, have aimed to mitigate this redundancy by serving as intermediaries that store and retrieve commonly used user data. The advent of large language models (LLMs) presents an opportunity to redefine memory management through an AI-native paradigm: SECOND ME. SECOND ME acts as an intelligent, persistent memory offload system that retains, organizes, and dynamically utilizes user-specific knowledge. By serving as an intermediary in user interactions, it can autonomously generate context-aware responses, prefill required information, and facilitate seamless communication with external systems, significantly reducing cognitive load and interaction friction. Unlike traditional memory storage solutions, SECOND ME extends beyond static data retention by leveraging LLM-based memory parameterization. This enables structured organization, contextual reasoning, and adaptive knowledge retrieval, facilitating a more systematic and intelligent approach to memory management. As AI-driven personal agents like SECOND ME become increasingly integrated into digital ecosystems, SECOND ME further represents a critical step toward augmenting human-world interaction with persistent, contextually aware, and self-optimizing memory systems. We have open-sourced the fully localizable deployment system at GitHub: https://github.com/Mindverse/Second-Me.

Figures (6)

• Figure 1: Hybrid Architecture of Second Me
• Figure 2: Automated Personal Model pipeline with LLM as a Judge and LLM as data synthesizer
• Figure 3: Given same query, here are three synthetic responses using different COT strategies.
• Figure 4: A concrete example (Case 1) from the context enhance task illustrating the superiority of Strong COT without DPO compared to Weak COT without DPO. The textbfd content represents the entities that exist in user's record.
• Figure 5: A concrete example (Case 2) from the context enhance task illustrating the superiority of Strong COT with DPO compared to Weak COT with DPO. The textbfd content represents the entities that exist in user's record.