Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Promoting Critical Thinking With Domain-Specific Generative AI Provocations

Thomas Şerban von Davier, Hao-Ping Lee, Jodi Forlizzi, Sauvik Das

Abstract

The evidence on the effects of generative AI (GenAI) on critical thinking is mixed, with studies suggesting both potential harms and benefits depending on its implementation. Some argue that AI-driven provocations, such as questions asking for human clarification and justification, are beneficial for eliciting critical thinking. Drawing on our experience designing and evaluating two GenAI-powered tools for knowledge work, ArtBot in the domain of fine art interpretation and Privy in the domain of AI privacy, we reflect on how design decisions shape the form and effectiveness of such provocations. Our observations and user feedback suggest that domain-specific provocations, implemented through productive friction and interactions that depend on user contribution, can meaningfully support critical thinking. We present participant experiences with both prototypes and discuss how supporting critical thinking may require moving beyond static provocations toward approaches that adapt to user preferences and levels of expertise.

Promoting Critical Thinking With Domain-Specific Generative AI Provocations

Abstract

The evidence on the effects of generative AI (GenAI) on critical thinking is mixed, with studies suggesting both potential harms and benefits depending on its implementation. Some argue that AI-driven provocations, such as questions asking for human clarification and justification, are beneficial for eliciting critical thinking. Drawing on our experience designing and evaluating two GenAI-powered tools for knowledge work, ArtBot in the domain of fine art interpretation and Privy in the domain of AI privacy, we reflect on how design decisions shape the form and effectiveness of such provocations. Our observations and user feedback suggest that domain-specific provocations, implemented through productive friction and interactions that depend on user contribution, can meaningfully support critical thinking. We present participant experiences with both prototypes and discuss how supporting critical thinking may require moving beyond static provocations toward approaches that adapt to user preferences and levels of expertise.
Paper Structure (9 sections, 2 figures, 1 table)

This paper contains 9 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Case Studies
  4. ArtBot: Domain-Grounded Provocation for Art Interpretation
  5. Privy: Structured Provocation for AI Privacy Planning
  6. Supporting Critical Thinking Through Challenging Provocations
  7. Discussion
  8. Conclusion
  9. System Screenshots

Figures (2)

  • Figure 1: ArtBot is an LLM-powered tool that challenges participants with questions to encourage them to share their own interpretations of the artwork in the shared workspace.
  • Figure 2: Privy is an LLM-powered tool that guides practitioners through structured privacy impact assessments to: (i) identify relevant risks in novel AI product concepts, and (ii) propose appropriate mitigations.