An Experimental Comparison of Cognitive Forcing Functions for Execution Plans in AI-Assisted Writing: Effects On Trust, Overreliance, and Perceived Critical Thinking

TL;DR

This work investigates plan-centered cognitive forcing functions (CFFs) for AI-generated execution plans in AI-assisted writing. By grounding CFFs in Halpern’s critical thinking framework and metacognitive monitoring, the study compares Assumptions, WhatIf, Both, and None across a large online experiment (n=214) and think-aloud interviews (n=12). The key finding is that Assumptions consistently reduces overreliance and supports calibrating trust without increasing cognitive load, while WhatIf is perceived as more helpful but yields weaker objective gains and higher mental effort. The results highlight plan-centered CFFs as a scalable design pattern for promoting critical reflection in GenAI-enabled knowledge work and suggest tailoring prompts to individual cognitive dispositions for maximum effect. The findings have practical implications for integrating lightweight, plan-focused reasoning prompts into AI tools to improve trust calibration and reduce automation bias in open-ended tasks like writing and planning.

Abstract

Generative AI (GenAI) tools improve productivity in knowledge workflows such as writing, but also risk overreliance and reduced critical thinking. Cognitive forcing functions (CFFs) mitigate these risks by requiring active engagement with AI output. As GenAI workflows grow more complex, systems increasingly present execution plans for user review. However, these plans are themselves AI-generated and prone to overreliance, and the effectiveness of applying CFFs to AI plans remains underexplored. We conduct a controlled experiment in which participants completed AI-assisted writing tasks while reviewing AI-generated plans under four CFF conditions: Assumption (argument analysis), WhatIf (hypothesis testing), Both, and a no-CFF control. A follow-up think-aloud and interview study qualitatively compared these conditions. Results show that the Assumption CFF most effectively reduced overreliance without increasing cognitive load, while participants perceived the WhatIf CFF as most helpful. These findings highlight the value of plan-focused CFFs for supporting critical reflection in GenAI-assisted knowledge work.

Figures (13)

• Figure 1: AI-assisted knowledge work with AI plan–based cognitive forcing functions. We illustrate a representative knowledge work task involving the analysis of product reviews. The interface presents a task description along with a lengthy task artifact comprising multiple product reviews. Upon clicking the "AI Assist Me" button, the system generates an AI execution plan and a corresponding draft. Before users can review the draft and decide whether to accept or reject the AI output, they are presented with cognitive forcing functions (CFFs) centered on the AI-generated plan and instantiated as short evaluative questions. After engaging with these questions, users proceed to the task review stage, where they assess the readiness of the AI output and provide follow-up instructions.
• Figure 2: Illustration of the cognitive forcing function (CFF) conditions evaluated in our study, shown for knowledge work Task 2 in (a). (b) shows the None condition, which comprises a simple "Proceed" button and takes users directly to the final assessment stage of the task. (c) shows the Assumptions condition, in which users are prompted to identify assumptions underlying a selected step of the AI-generated plan (illustrated here for Step 5). Users may explore multiple plan steps, but must respond to at least one prompt before proceeding to the final assessment stage. (d) shows the WhatIf condition, which consists of two prompts: the first asks users to identify what they perceive as the most critical step in the plan, and the second asks them to reason about how the outcome might change if that step were to fail or be altered. In both the Assumptions and WhatIf conditions, there are no correct or incorrect answers; the prompts are designed to encourage critical reflection rather than correctness. In the Both condition, users first engage with the Assumptions prompt and then with the WhatIf prompt before proceeding to the final assessment stage. Additional details are provided in Section \ref{['sec:method.cff']}.
• Figure 3: Overview of study procedures for the online unmoderated study and interview conditions. Across both study variants, participants first completed a presurvey, then engaged in a sequence of AI-assisted knowledge work tasks (T1–T5) with a specific CFF condition, and finally completed a postsurvey. The presurvey included questions on participant demographics, experience with generative AI tools, need for cognition, and actively open-minded thinking. The postsurvey included questions on the participants' overall study experience, their perceived critical thinking while engaging with different cognitive forcing function (CFF) conditions, and their sense of agency when completing the tasks. Each study session lasted approximately one hour, with the breakdown of time across phases shown in the corresponding figures. Panel (a) shows the detailed pipeline for the online unmoderated study, panel (b) shows the pipeline for the interview study, and panel (c) shows the interaction flow within a single AI-assisted knowledge work task.
• Figure 4: Distribution of participants' need for cognition scores, thinking disposition measured via the actively open-minded thinking scale, and their self-reported usage of and trust in GenAI tools collected via the presurvey.
• Figure 5: Demographic distribution of participants by age and work industry in the online comparative experiment.