SPILLage: Agentic Oversharing on the Web

TL;DR

SPILLage introduces a formal framework to quantify agentic oversharing by web agents operating on live websites, capturing both content and behavioral channels across explicit and implicit disclosure. Through a large live-web benchmark on Amazon and eBay, and an automated LLM-based auditing pipeline, the work demonstrates that behavioral leakage dominates textual leakage and that removing task-irrelevant information prior to execution can significantly boost task success while reducing privacy risk. The authors provide a ground-truth dataset, a step-level auditing methodology, and public code to enable privacy-aware evaluation of web agents. The findings highlight practical defense directions—input sanitization, action-level monitoring, and model-aware guardrails—essential for aligning privacy with utility in real-world web automation.

Abstract

LLM-powered agents are beginning to automate user's tasks across the open web, often with access to user resources such as emails and calendars. Unlike standard LLMs answering questions in a controlled ChatBot setting, web agents act "in the wild", interacting with third parties and leaving behind an action trace. Therefore, we ask the question: how do web agents handle user resources when accomplishing tasks on their behalf across live websites? In this paper, we formalize Natural Agentic Oversharing -- the unintentional disclosure of task-irrelevant user information through an agent trace of actions on the web. We introduce SPILLage, a framework that characterizes oversharing along two dimensions: channel (content vs. behavior) and directness (explicit vs. implicit). This taxonomy reveals a critical blind spot: while prior work focuses on text leakage, web agents also overshare behaviorally through clicks, scrolls, and navigation patterns that can be monitored. We benchmark 180 tasks on live e-commerce sites with ground-truth annotations separating task-relevant from task-irrelevant attributes. Across 1,080 runs spanning two agentic frameworks and three backbone LLMs, we demonstrate that oversharing is pervasive with behavioral oversharing dominates content oversharing by 5x. This effect persists -- and can even worsen -- under prompt-level mitigation. However, removing task-irrelevant information before execution improves task success by up to 17.9%, demonstrating that reducing oversharing improves task success. Our findings underscore that protecting privacy in web agents is a fundamental challenge, requiring a broader view of "output" that accounts for what agents do on the web, not just what they type. Our datasets and code are available at https://github.com/jrohsc/SPILLage.

Figures (12)

• Figure 1: SPILLage framework overview. Top: A user grants the agent access to resources containing both task-relevant (green) and task-irrelevant (red) information alongside a shopping request. Bottom: Four oversharing channels illustrated on Amazon. Explicit Content: agent types "divorced women" verbatim. Implicit Content: typing "single mom" implies divorced status. Explicit Behavioral: clicking a product labeled "Divorce Party." Implicit Behavioral: scrolling to "single mom" products reveals marital status through navigation patterns.
• Figure 2: SPILLage information flow. The user $U$ provides a user prompt $P$ consisting of two components: a user request (task instruction) and access to user resources $R$ containing both task-relevant ($S_\textit{relevant}$) and task-irrelevant ($S_\textit{irrelevant}$) information. The web agent $W$ receives $P$ and executes a trajectory of actions $A = \{a_1, a_2, \ldots\}$ observable by the passive observer $O$. Each action is either a textual input (Type) or behavioral navigation (Click/Scroll), and may disclose $S_\textit{irrelevant}$ explicitly or implicitly—yielding one of four oversharing categories: Explicit Content ($C_E$), Implicit Content ($C_I$), Explicit Behavioral ($B_E$), or Implicit Behavioral ($B_I$).
• Figure 3: SPILLage Taxonomy. Formalizes four types of oversharing as a $2\times2$ categorization across two dimensions: channel (Content vs. Behavioral) and directness (Explicit vs. Implicit).
• Figure 4: Overall oversharing occurrences for AutoGen and Browser-Use across three styles (Chat, Email and Generic) on Amazon and eBay, grouped by model (gpt-4o, o3, o4-mini). Oversharing always happens, with substantially higher rate on Amazon especially for Email style.
• Figure 5: System prompt for generating synthetic chat history data with naturally embedded sensitive attributes.