"AI enhances our performance, I have no doubt this one will do the same": The Placebo effect is robust to negative descriptions of AI

TL;DR

The study shows that expectations about AI can yield placebo-like improvements in performance during human-AI interaction, and these effects persist even when descriptions are framed negatively. Using a letter-discrimination task and Bayesian cognitive modeling, the authors demonstrate AI performance bias: participants perform better and with faster information processing when they believe an AI is aiding them, largely independent of verbal descriptions. A Drift Diffusion Model reveals that AI presence increases drift rate $\nu$ (faster evidence accumulation) and can induce a more conservative boundary $\alpha$, while workload and arousal measures show no consistent changes. A replication study confirms the robustness of the bias, and qualitative analyses reveal that AI narratives and user beliefs shape expectations and decision strategies. Collectively, the work provides a behavioral marker for placebo effects in human-AI interaction and argues for accounting for AI narratives in evaluating AI-enabled systems.

Abstract

Heightened AI expectations facilitate performance in human-AI interactions through placebo effects. While lowering expectations to control for placebo effects is advisable, overly negative expectations could induce nocebo effects. In a letter discrimination task, we informed participants that an AI would either increase or decrease their performance by adapting the interface, but in reality, no AI was present in any condition. A Bayesian analysis showed that participants had high expectations and performed descriptively better irrespective of the AI description when a sham-AI was present. Using cognitive modeling, we could trace this advantage back to participants gathering more information. A replication study verified that negative AI descriptions do not alter expectations, suggesting that performance expectations with AI are biased and robust to negative verbal descriptions. We discuss the impact of user expectations on AI interactions and evaluation and provide a behavioral placebo marker for human-AI interaction

Figures (7)

• Figure 1: Trial sequence during the letter discrimination task. The duration of the ISI followed a Gaussian distribution (M = 1000 ms, SD = 600 ms). Key responses (left or right arrow) were logged during the presentation of the mask.
• Figure 2: Study Procedure: In this mixed-design study examining the induction of placebo and nocebo effects, participants were divided into two groups (Description), with each group receiving altered system descriptions (negative: AI decreased task performance and increased stress in users/ positive: AI increased task performance and decreased stress in users). Participants in each group performed a letter discrimination task under two conditions (system statusStatus): in the sham-AI (sAI) activesham-AI condition, they were informed that an AI system was active and adjusting the task pace based on their measured stress responsesthe task pace was adjusted by an AI system based on their measured stress responses; in the sAI inactiveno-AI condition, they were told that the AI system was inactive and adjustments in task pace were random. The order of system status alternated within each description group. Before and after interacting with the sAIsham-AI system, expectations on performance with and without the sAIsham-AI system set as active were assessed. After the tasks and before debriefing, additional questionnaires assessing i.e., user experience and AI literacy were implemented. Ultimately, we revealed the manipulation and assessed the participants' belief in the manipulation.
• Figure 3: The participants interacted with the system with their dominant hand using a keyboard and a mouse.
• Figure 4: A: Mean expected performance as a function of Time and Description. B: Mean expected relative speed as a function of Time and Description. C: Mean expected correct responses before and after interacting with the sAIsham-AI system as a function of Time, System Status and Description. Error bars denote +-1 standard error of the mean.
• Figure 5: A: Reaction time distribution as a function of System Statusstatus (sAI active vs. sAI inactive) and Description (incorrect trials are multiplied with -1). B: Posterior density plot for the parameter values for all population-level parameters 95% High-Density Interval (HDI). If the HDI does not cross the midline $p_b$ will be <2.5%.