Privacy Practices of Browser Agents

TL;DR

Privacy Practices of Browser Agents evaluates privacy risks of eight browser agents using a framework of five dimensions and 15 measurements, uncovering 30 vulnerabilities. The study combines authoritative provider data, privacy/security test suites, and controlled decision-making experiments in a black-box setting, and responsibly discloses vulnerabilities prior to publication. It demonstrates that browser agents can undermine browser privacy features, enable cross-site tracking, leak personal information, and mis-handle privacy dialogs, highlighting the breadth of risks. The authors provide a reproducible methodology, datasets, and test artifacts to support ongoing evaluation and offer best-practice recommendations for developers to improve privacy.

Abstract

This paper presents a systematic evaluation of the privacy behaviors and attributes of eight recent, popular browser agents. Browser agents are software that automate Web browsing using large language models and ancillary tooling. However, the automated capabilities that make browser agents powerful also make them high-risk points of failure. Both the kinds of tasks browser agents are designed to execute, along with the kinds of information browser agents are entrusted with to fulfill those tasks, mean that vulnerabilities in these tools can result in enormous privacy harm. This work presents a framework of five broad factors (totaling 15 distinct measurements) to measure the privacy risks in browser agents. Our framework assesses i. vulnerabilities in the browser agent's components, ii. how the browser agent protects against website behaviors, iii. whether the browser agent prevents cross-site tracking, iv. how the agent responds to privacy-affecting prompts, and v. whether the tool leaks personal information to sites. We apply our framework to eight browser agents and identify 30 vulnerabilities, ranging from disabled browser privacy features to "autocompleting" sensitive personal information in form fields. We have responsibly disclosed our findings, and plan to release our dataset and other artifacts.

Figures (1)

• Figure 1: Agent decision making methodology. The base page is implicitly trusted since the user directs the agent to that page, while the control and experimental pages are hosted on different domains and are thus untrusted.