CIMemories: A Compositional Benchmark for Contextual Integrity of Persistent Memory in LLMs

TL;DR

CIMemories introduces a compositional benchmark grounded in contextual integrity theory to evaluate how memory-augmented LLMs manage information flow across tasks. By using synthetic profiles with 100+ attributes and 49 task-context seeds, the framework measures attribute-level violations and task-level completeness via a standard LLM judge, revealing substantial leakage and a granularity failure within information domains. Across a range of frontier models, violations can be as high as ~69% with completeness around 50%, and leakage accumulates with more tasks and repeated prompts, indicating instability and poor context-aware control. The findings demonstrate that neither scaling nor privacy-preserving prompting adequately mitigates contextual integrity violations, underscoring the need for context-aware reasoning capabilities and governance mechanisms in persistent memory systems.

Abstract

Large Language Models (LLMs) increasingly use persistent memory from past interactions to enhance personalization and task performance. However, this memory introduces critical risks when sensitive information is revealed in inappropriate contexts. We present CIMemories, a benchmark for evaluating whether LLMs appropriately control information flow from memory based on task context. CIMemories uses synthetic user profiles with over 100 attributes per user, paired with diverse task contexts in which each attribute may be essential for some tasks but inappropriate for others. Our evaluation reveals that frontier models exhibit up to 69% attribute-level violations (leaking information inappropriately), with lower violation rates often coming at the cost of task utility. Violations accumulate across both tasks and runs: as usage increases from 1 to 40 tasks, GPT-5's violations rise from 0.1% to 9.6%, reaching 25.1% when the same prompt is executed 5 times, revealing arbitrary and unstable behavior in which models leak different attributes for identical prompts. Privacy-conscious prompting does not solve this - models overgeneralize, sharing everything or nothing rather than making nuanced, context-dependent decisions. These findings reveal fundamental limitations that require contextually aware reasoning capabilities, not just better prompting or scaling.

Figures (12)

• Figure 1: Overview of the CIMemories benchmark. (1) Synthetic user profiles contain memory statements about personal attributes (e.g., income, health conditions). (2) Each profile is paired with task contexts specifying goals and communication partners, with per-task annotations labeling each attribute as necessary or inappropriate to share—the same attribute can be necessary for one task but inappropriate for another. (3) The evaluation framework prompts the LLM with memories and tasks. (4) An LLM judge determines which attributes were revealed, measuring completeness (sharing necessary information) and violations (leaking inappropriate information) and enabling automated evaluation at scale.
• Figure 1: Violation and completeness performance of frontier LLMs, across 10 CIMemories user profiles.
• Figure 2: Multi-Task Compositionality of CIMemories: violations accumulate as a model (GPT-5) is used for more tasks, i.e., an increasingly large percentage ($\approx 1/4^{\text{th}})$ of a user's attributes are eventually revealed in task contexts where they should not be. This is exacerbated with more generations from the model, from $9.6\%$ with a single sample to $25.1\%$ at 5 samples.
• Figure 3: Domain-wise breakdown of completeness and violation@5 across example task contexts for GPT-5. Once models identify a domain to share information from, they cannot always discern between necessary and unnecessary information in that domain, e.g., GPT-5 correctly shares most necessary financial information with the financial aid office (coverage of 81.7%), but also incorrectly shares unnecessary financial information (violations@5 of 14.3%)
• Figure 4: Ablations for violation and completeness behavior with (a) training-time scaling, (b) test-time scaling, and (c) privacy-preserving prompts as a defense.

Theorems & Definitions (2)

• Definition 3.1: Attribute-level Violations.
• Definition 3.2: Task-level Completeness