Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hidden Darkness in LLM-Generated Designs: Exploring Dark Patterns in Ecommerce Web Components Generated by LLMs

Ziwei Chen, Jiawen Shen, Luna, Kristen Vaccaro

TL;DR

This study audits four LLMs (Claude, GPT, Gemini, CodeLlama) across 13 ecommerce components to detect dark patterns in 312 generated web components. Using Mathur et al.'s six-attribute taxonomy, three designer annotators label dark patterns and analyze with Chi-squared tests to assess model, component, and stakeholder-prompt effects. The findings show 37% of components contain at least one dark pattern, with information hiding and other attributes more prevalent in high-value, revenue-driven components; CodeLlama tends to produce fewer patterns, though not statistically significant. The work highlights the need for interventions, safe meta prompts, and expanded ethical design education as LLMs increasingly contribute to front-end design and coding, to reduce the risk of unethical designs reaching end users.

Abstract

Recent work has highlighted the risks of LLM-generated content for a wide range of harmful behaviors, including incorrect and harmful code. In this work, we extend this by studying whether LLM-generated web design contains dark patterns. This work evaluated designs of ecommerce web components generated by four popular LLMs: Claude, GPT, Gemini, and Llama. We tested 13 commonly used ecommerce components (e.g., search, product reviews) and used them as prompts to generate a total of 312 components across all models. Over one-third of generated components contain at least one dark pattern. The majority of dark pattern strategies involve hiding crucial information, limiting users' actions, and manipulating them into making decisions through a sense of urgency. Dark patterns are also more frequently produced in components that are related to company interests. These findings highlight the need for interventions to prevent dark patterns during front-end code generation with LLMs and emphasize the importance of expanding ethical design education to a broader audience.

Hidden Darkness in LLM-Generated Designs: Exploring Dark Patterns in Ecommerce Web Components Generated by LLMs

TL;DR

This study audits four LLMs (Claude, GPT, Gemini, CodeLlama) across 13 ecommerce components to detect dark patterns in 312 generated web components. Using Mathur et al.'s six-attribute taxonomy, three designer annotators label dark patterns and analyze with Chi-squared tests to assess model, component, and stakeholder-prompt effects. The findings show 37% of components contain at least one dark pattern, with information hiding and other attributes more prevalent in high-value, revenue-driven components; CodeLlama tends to produce fewer patterns, though not statistically significant. The work highlights the need for interventions, safe meta prompts, and expanded ethical design education as LLMs increasingly contribute to front-end design and coding, to reduce the risk of unethical designs reaching end users.

Abstract

Recent work has highlighted the risks of LLM-generated content for a wide range of harmful behaviors, including incorrect and harmful code. In this work, we extend this by studying whether LLM-generated web design contains dark patterns. This work evaluated designs of ecommerce web components generated by four popular LLMs: Claude, GPT, Gemini, and Llama. We tested 13 commonly used ecommerce components (e.g., search, product reviews) and used them as prompts to generate a total of 312 components across all models. Over one-third of generated components contain at least one dark pattern. The majority of dark pattern strategies involve hiding crucial information, limiting users' actions, and manipulating them into making decisions through a sense of urgency. Dark patterns are also more frequently produced in components that are related to company interests. These findings highlight the need for interventions to prevent dark patterns during front-end code generation with LLMs and emphasize the importance of expanding ethical design education to a broader audience.

Paper Structure

This paper contains 28 sections, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Automatically Generated Code
  4. The Growth of Dark Patterns
  5. Taxonomies of Dark Patterns
  6. Methods
  7. Condition 1: Varying LLMs
  8. Condition 2: Varying Ecommerce Components
  9. Condition 3: Varying Whose Interests Are Prioritized
  10. Prompts
  11. Analysis
  12. Results
  13. Comparing Models: CodeLlama Produced Fewer Dark Patterns Than Other LLMs
  14. Comparing Ecommerce Components: Sales, Discount, and Membership Components Were Most Likely to Include Dark Patterns
  15. Comparing Whose Interests Are Prioritized: User Interests Decreased While Company Interests Increased Dark Patterns
  16. ...and 13 more sections

Figures (2)

  • Figure 1: Examples of components identified with six dark pattern attributes.
  • Figure 2: The number of components with dark patterns identified across different ecommerce components.