Automatically Detecting Online Deceptive Patterns

TL;DR

AutoBot tackles the pervasive problem of online deceptive patterns by introducing a modular, screenshot-based framework that localizes and classifies deceptive UI patterns without relying on HTML. It decouples the task into a Vision Module that constructs an ElementMap from a webpage screenshot and a Language Module that reasons over this map with a taxonomy of static deceptive patterns, using a distillation pipeline to enable cost-effective, privacy-preserving deployment on smaller LMs. A synthetic-data strategy (teacher-student distillation) enables high-accuracy classification with small models, achieving near-perfect performance on the evaluation dataset and strong end-to-end results, while remaining practical for browser extensions, Lighthouse audits, and web-scale measurements. The work demonstrates real-world applicability through three applications, reports broad measurements across thousands of sites, and provides user and developer studies to validate usability and impact, establishing AutoBot as a scalable tool for mitigating deceptive patterns on the web.

Abstract

Deceptive patterns in digital interfaces manipulate users into making unintended decisions, exploiting cognitive biases and psychological vulnerabilities. These patterns have become ubiquitous on various digital platforms. While efforts to mitigate deceptive patterns have emerged from legal and technical perspectives, a significant gap remains in creating usable and scalable solutions. We introduce our AutoBot framework to address this gap and help web stakeholders navigate and mitigate online deceptive patterns. AutoBot accurately identifies and localizes deceptive patterns from a screenshot of a website without relying on the underlying HTML code. AutoBot employs a two-stage pipeline that leverages the capabilities of specialized vision models to analyze website screenshots, identify interactive elements, and extract textual features. Next, using a large language model, AutoBot understands the context surrounding these elements to determine the presence of deceptive patterns. We also use AutoBot, to create a synthetic dataset to distill knowledge from 'teacher' LLMs to smaller language models. Through extensive evaluation, we demonstrate AutoBot's effectiveness in detecting deceptive patterns on the web, achieving an F1-score of 0.93 when detecting deceptive patterns, underscoring its potential as an essential tool for mitigating online deceptive patterns. We implement AutoBot, across three downstream applications targeting different web stakeholders: (1) a local browser extension providing users with real-time feedback, (2) a Lighthouse audit to inform developers of potential deceptive patterns on their sites, and (3) as a measurement tool designed for researchers and regulators.

Figures (14)

• Figure 1: OpenAI's GPT4.5 incorrectly identifies the color of the "Reject All" button as being less prominent than the other, leading to an incorrect classification of "nudge". Similarly, Gemini 2.5 Pro incorrectly notes that the two buttons in the cookie notices are visually distinct from each other, resulting in a misclassification of "nudge".
• Figure 2: Taxonomy of Deceptive Patterns. AutoBot classifies text elements into five high-level deceptive pattern categories: Interface Interference, Obstruction, Forced Action, Sneaking, and Non-Deceptive.
• Figure 3: Overview of AutoBot's working process. AutoBot takes a screenshot through a multi-stage framework consisting of the Vision Module and the Language Module, and returns the Deceptive Pattern classification, Subtype, and reasoning for each element in the screenshot. The results are then used by applications such as browser extensions and Lighthouse.
• Figure 4: Pipeline of Generating Web-UI Element Dataset to train YOLOv10. We used GPT-4 to generate 2.5K ideas (Idea Datasets), which were then processed by v0 to create 7.5K websites (Synthetic Website Dataset). After manually verifying these sites for rendering errors and randomizing their UI library, we capture over 60K screenshots to train our YOLOv10 model.
• Figure 5: The input and output structure of our language module. The input ElementMap consists of key features of a web element, and the output contains a deceptive category, subtype, and reasoning of classification.