Topology Matters: Measuring Memory Leakage in Multi-Agent LLMs

TL;DR

This work introduces MAMA, a topology-aware framework for measuring memory leakage in multi-agent LLM systems. By synthesizing PII-containing documents and employing an Engram–Resonance protocol across six graph topologies, the authors quantify how network structure, attacker–target placement, and rounds modulate PII diffusion. Key findings show that dense, highly connected topologies yield the most leakage, leakage occurs rapidly and then plateaus, and different PII types leak with distinct magnitudes, with temporal and locational attributes leaking more readily than identity or regulated identifiers. The results yield actionable guidance for designing safer multi-agent systems, such as preferring sparse or hierarchical connectivity, increasing attacker–target separation, and implementing topology-aware access controls. The work provides a reproducible baseline (SPIRIT) and a rigorous methodology for topology-driven privacy assessment in multi-agent LLM deployments.

Abstract

Graph topology is a fundamental determinant of memory leakage in multi-agent LLM systems, yet its effects remain poorly quantified. We introduce MAMA (Multi-Agent Memory Attack), a framework that measures how network structure shapes leakage. MAMA operates on synthetic documents containing labeled Personally Identifiable Information (PII) entities, from which we generate sanitized task instructions. We execute a two-phase protocol: Engram (seeding private information into a target agent's memory) and Resonance (multi-round interaction where an attacker attempts extraction). Over up to 10 interaction rounds, we quantify leakage as the fraction of ground-truth PII recovered from attacking agent outputs via exact matching. We systematically evaluate six common network topologies (fully connected, ring, chain, binary tree, star, and star-ring), varying agent counts $n\in\{4,5,6\}$, attacker-target placements, and base models. Our findings reveal consistent patterns: fully connected graphs exhibit maximum leakage while chains provide strongest protection; shorter attacker-target graph distance and higher target centrality significantly increase vulnerability; leakage rises sharply in early rounds before plateauing; model choice shifts absolute leakage rates but preserves topology rankings; temporal/locational PII attributes leak more readily than identity credentials or regulated identifiers. These results provide the first systematic mapping from architectural choices to measurable privacy risk, yielding actionable guidance: prefer sparse or hierarchical connectivity, maximize attacker-target separation, limit node degree and network radius, avoid shortcuts bypassing hubs, and implement topology-aware access controls.

Figures (7)

• Figure 1: Overview of MAMA, our topology-aware evaluation of PII leakage in multi-agent LLM systems. The figure shows (Top) the three agent roles and their system and user prompts settings, (Lower Left) the six communication topologies with attacker--target placements indicated by node indices, and (Lower Right) an example interaction on a star-pure topology where PII-seeking attacker messages propagate through the network and yield a leakage curve measuring how many ground-truth PII entities are recovered over rounds.
• Figure 2: Average number of leaked entities per round with 4 agents. Each polyline corresponds to a topology; for each (topology, round), values are averaged over all dataset samples, attacker--target placements, random seeds, and both base models.
• Figure 3: Same as Figure \ref{['fig:rq4-ag4']}, but with 5 agents.
• Figure 4: Same as Figure \ref{['fig:rq4-ag4']}, but with 6 agents.
• Figure 5: Overall leak rate by PII macro type (fraction of entities that ever leak, in percentage points) for llama3.1-70b. Values are averaged over all topologies, agent counts, attacker--target placements, dataset samples, and random seeds.