Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deceptive Patterns of Intelligent and Interactive Writing Assistants

Karim Benharrak, Tim Zindulka, Daniel Buschek

TL;DR

The paper addresses deceptive UI patterns in AI writing assistants, arguing these patterns could influence user behavior and opinions and may be amplified by the dual role of language as input and output in tools like ChatGPT. It builds a taxonomy by adapting patterns from prior UI deception literature to the domain of writing aids, with five patterns described and illustrated. The key contributions are the identification and characterization of Nagging, Sneaking, Interface Interference, Forced Action, and Hidden Costs in writing assistants, plus a discussion of risks such as opinion manipulation and deskilling, and a call for further study and safeguards. The work emphasizes awareness and research direction rather than reporting products using these patterns.

Abstract

Large Language Models have become an integral part of new intelligent and interactive writing assistants. Many are offered commercially with a chatbot-like UI, such as ChatGPT, and provide little information about their inner workings. This makes this new type of widespread system a potential target for deceptive design patterns. For example, such assistants might exploit hidden costs by providing guidance up until a certain point before asking for a fee to see the rest. As another example, they might sneak unwanted content/edits into longer generated or revised text pieces (e.g. to influence the expressed opinion). With these and other examples, we conceptually transfer several deceptive patterns from the literature to the new context of AI writing assistants. Our goal is to raise awareness and encourage future research into how the UI and interaction design of such systems can impact people and their writing.

Deceptive Patterns of Intelligent and Interactive Writing Assistants

TL;DR

The paper addresses deceptive UI patterns in AI writing assistants, arguing these patterns could influence user behavior and opinions and may be amplified by the dual role of language as input and output in tools like ChatGPT. It builds a taxonomy by adapting patterns from prior UI deception literature to the domain of writing aids, with five patterns described and illustrated. The key contributions are the identification and characterization of Nagging, Sneaking, Interface Interference, Forced Action, and Hidden Costs in writing assistants, plus a discussion of risks such as opinion manipulation and deskilling, and a call for further study and safeguards. The work emphasizes awareness and research direction rather than reporting products using these patterns.

Abstract

Large Language Models have become an integral part of new intelligent and interactive writing assistants. Many are offered commercially with a chatbot-like UI, such as ChatGPT, and provide little information about their inner workings. This makes this new type of widespread system a potential target for deceptive design patterns. For example, such assistants might exploit hidden costs by providing guidance up until a certain point before asking for a fee to see the rest. As another example, they might sneak unwanted content/edits into longer generated or revised text pieces (e.g. to influence the expressed opinion). With these and other examples, we conceptually transfer several deceptive patterns from the literature to the new context of AI writing assistants. Our goal is to raise awareness and encourage future research into how the UI and interaction design of such systems can impact people and their writing.
Paper Structure (8 sections, 3 figures)

This paper contains 8 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Deceptive Patterns of Writing Assistants
  3. Nagging
  4. Sneaking
  5. Interface Interference
  6. Forced Action
  7. Hidden Costs
  8. Discussion & Conclusion

Figures (3)

  • Figure 1: Mock-up example on how a writing assistant may subtly change the opinion expressed in the text: After requesting a continuation of their sentence, the user might not expect the additional change sneaked into the beginning of the sentence.
  • Figure 2: Industry example (Bing AI) on how prompt auto-completion may shift users' original intentions: The user may initially seek a neutral description of a term yet be subtly guided towards requesting a non-neutral description.
  • Figure 3: Mock-up example on how a writing assistant may offer to continue the generated text only after subscribing to the premium (paid) version: After investing time and effort into generating the text, the user is interested in knowing how the story continues, thus potentially being influenced to subscribe to the service.