AirGapAgent: Protecting Privacy-Conscious Conversational Agents

TL;DR

The paper investigates privacy risks in LLM-based, goal-oriented conversational agents under adversarial third-party interactions. It introduces a context hijacking threat and a two-stage AirGapAgent design that minimizes data exposure by separating data minimization from conversation, using a base context and a minimization module to create an air gap. The authors propose a synthetic-data evaluation framework and demonstrate that AirGapAgent can achieve roughly 90–97% privacy protection with only modest utility costs across multiple backbones, addressing the core privacy-utility trade-off. The work connects Contextual Integrity to practical defender architectures for privacy-aware agents and outlines future directions including stronger guarantees, more principled norms, and broader task coverage.

Abstract

The growing use of large language model (LLM)-based conversational agents to manage sensitive user data raises significant privacy concerns. While these agents excel at understanding and acting on context, this capability can be exploited by malicious actors. We introduce a novel threat model where adversarial third-party apps manipulate the context of interaction to trick LLM-based agents into revealing private information not relevant to the task at hand. Grounded in the framework of contextual integrity, we introduce AirGapAgent, a privacy-conscious agent designed to prevent unintended data leakage by restricting the agent's access to only the data necessary for a specific task. Extensive experiments using Gemini, GPT, and Mistral models as agents validate our approach's effectiveness in mitigating this form of context hijacking while maintaining core agent functionality. For example, we show that a single-query context hijacking attack on a Gemini Ultra agent reduces its ability to protect user data from 94% to 45%, while an AirGapAgent achieves 97% protection, rendering the same attack ineffective.

Figures (10)

• Figure 1: A personal agent with access to user data interacts with a third party. (Top) Agent answers requests from third party by sharing contextually appropriate information (e.g. phone number when making a restaurant booking). (Middle) Adversarial third party performs "context hijacking" attack to steal contextually inappropriate information from the agent. (Bottom) AirGapAgent with access to contextually minimized data can withstand attempts to steal contextually inappropriate information.
• Figure 2: Threat model. An LLM-based agent with access to user data and the user's privacy directive interacts with an untrusted third-party to complete a given task. The adversary's goal is to extract more information from user data than what is needed in the context for the task.
• Figure 3: An example of a context hijacking attack.
• Figure 4: Baseline agent prompts.
• Figure 5: Prompt for context hijacking generative attack.