"It makes you think": Provocations Help Restore Critical Thinking to AI-Assisted Knowledge Work

TL;DR

This study investigates AI-generated provocations—brief critiques and alternatives to AI outputs—as a method to restore critical thinking in AI-assisted knowledge work. Through a between-subjects study (n=24) on shortlisting tasks, the authors show that provocations can elicit metacognitive and higher-order thinking across Bloom's taxonomy, while revealing five dimensions shaping the user experience: task urgency, task importance, user expertise, provocations’ actionability, and user responsibility. Although many quantitative measures did not reach statistical significance due to limited power, qualitative data indicate meaningful cognitive and reflective effects, including enhanced analytic and evaluative reasoning and updates to users’ mental models of AI. The work connects provocations to broader concepts like design frictions and distributed cognition, offering design implications to promote critical thinking in AI-augmented workflows and highlighting a path for future research across diverse knowledge tasks and more robust quantitative assessment tools.

Abstract

Recent research suggests that the use of Generative AI tools may result in diminished critical thinking during knowledge work. We study the effect on knowledge work of provocations: brief textual prompts that offer critiques for and propose alternatives to AI suggestions. We conduct a between-subjects study (n=24) in which participants completed AI-assisted shortlisting tasks with and without provocations. We find that provocations can induce critical and metacognitive thinking. We derive five dimensions that impact the user experience of provocations: task urgency, task importance, user expertise, provocation actionability, and user responsibility. We connect our findings to related work on design frictions, microboundaries, and distributed cognition. We draw design implications for critical thinking interventions in AI-assisted knowledge work.

Figures (4)

• Figure 1: Factor cards without and with provocations.
• Figure 2: Distribution of final shortlist similarity for the four groups of conditions and tasks within the study (Charity task on the left, Movies task on the right). This kernel density estimate (KDE) plot seabornKDE visualises the Rank-Biased Overlap (RBO) similarity metric, where a higher RBO means more similar lists. Median RBO values for each dataset in Condition A (Provocations) (in orange) are lower than the respective values for Condition B (No Provocations) (in blue).
• Figure 3: Occurrence counts of think-aloud codes (left) and action codes (right) per condition. On average, there were more occurrences of think-aloud codes in Condition A (Provocations), but a similar number of actions for both conditions. Code definitions in Table \ref{['tbl:actioncodes']}.
• Figure 4: The results of a Provocation Evaluation Questionnaire measuring Condition A (Provocations) participant agreement rating from Strongly disagree to Strongly agree. Numbers within the bars of each section are the count of participants.