MetaMem: Evolving Meta-Memory for Knowledge Utilization through Self-Reflective Symbolic Optimization

TL;DR

MetaMem addresses fragmentation and misalignment in external memory for LLMs by introducing a self-evolving meta-memory that learns how to use memorized knowledge across tasks. The framework combines a memory module with a task-agnostic meta-memory $\\mathcal{E}_T$, which is updated through self-reflection and action-based optimization, yielding improved reasoning and knowledge utilization as $y = \\text{LLM}(\\text{Instruct}_{Gen}(q, \\mathcal{M}, \\mathcal{E}_T))$ and memory updates $\\mathcal{E}_{t+1} = \\texttt{Exec}(\\tilde{O}_t, \\mathcal{E}_t)$. Empirical results show MetaMem outperforms baselines by over $3.6\%$, generalizes across domains, and remains scalable with data, indicating strong potential for enhancing long-horizon human-LLM interactions. The work highlights the importance of learning how to utilize stored knowledge, not just storing it, and provides a principled, meta-learning-driven mechanism for memory-guided reasoning.

Abstract

Existing memory systems enable Large Language Models (LLMs) to support long-horizon human-LLM interactions by persisting historical interactions beyond limited context windows. However, while recent approaches have succeeded in constructing effective memories, they often disrupt the inherent logical and temporal relationships within interaction sessions, resulting in fragmented memory units and degraded reasoning performance. In this paper, we propose MetaMem, a novel framework that augments memory systems with a self-evolving meta-memory, aiming to teach LLMs how to effectively utilize memorized knowledge. During meta-memory optimization, MetaMem iteratively distills transferable knowledge utilization experiences across different tasks by self-reflecting on reasoning processes and performing actions to update the current meta-memory state. The accumulated meta-memory units serve as explicit knowledge utilization experiences, guiding the LLM to systematically identify and integrate critical evidence from scattered memory fragments. Extensive experiments demonstrate the effectiveness of MetaMem, which significantly outperforms strong baselines by over 3.6%. All codes and datasets are available at https://github.com/OpenBMB/MetaMem.

Figures (16)

• Figure 1: Illustration of Our MetaMem Framework. Green indicates correct evidence, while red represents distracting evidence. MetaMem guides the LLM to effectively utilize knowledge from scattered memory fragments, thereby generating the correct answer.
• Figure 2: Overview of MetaMem Model. MetaMem evolves through environmental feedback, guiding the memory system to utilize factual knowledge through Meta Memory.
• Figure 3: Generalization Ability of Meta-Memory. Performance is reported under Qwen3-30B-A3B-Instruct and Llama3.1-70B-Instruct backbone models.
• Figure 4: Performance of the Evolved Meta-Memory Across Different Training Steps.
• Figure 5: Effectiveness of MetaMem Optimized with Different Data Scales. Figure \ref{['fig:scaling:a']} illustrates the model performance as the training dataset size increases. Figure \ref{['fig:scaling:b']} reports the perplexity of model outputs conditioned on the learned meta-memory.