Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

When Two Wrongs Don't Make a Right" -- Examining Confirmation Bias and the Role of Time Pressure During Human-AI Collaboration in Computational Pathology

Emely Rosbach, Jonas Ammeling, Sebastian Krügel, Angelika Kießig, Alexis Fritz, Jonathan Ganz, Chloé Puget, Taryn Donovan, Andrea Klang, Maximilian C. Köller, Pompei Bolfa, Marco Tecilla, Daniela Denk, Matti Kiupel, Georgios Paraschou, Mun Keong Kok, Alexander F. H. Haake, Ronald R. de Krijger, Andreas F. -P. Sonnen, Tanit Kasantikul, Gerry M. Dorrestein, Rebecca C. Smedley, Nikolas Stathonikos, Matthias Uhl, Christof A. Bertram, Andreas Riener, Marc Aubreville

TL;DR

It is suggested that AI integration may fuel confirmation bias, evidenced by a statistically significant positive linear-mixed-effects model coefficient linking AI recommendations mirroring flawed human judgment and alignment with system advice, and time pressure appeared to weaken this relationship.

Abstract

Artificial intelligence (AI)-based decision support systems hold promise for enhancing diagnostic accuracy and efficiency in computational pathology. However, human-AI collaboration can introduce and amplify cognitive biases, such as confirmation bias caused by false confirmation when erroneous human opinions are reinforced by inaccurate AI output. This bias may worsen when time pressure, ubiquitously present in routine pathology, strains practitioners' cognitive resources. We quantified confirmation bias triggered by AI-induced false confirmation and examined the role of time constraints in a web-based experiment, where trained pathology experts (n=28) estimated tumor cell percentages. Our results suggest that AI integration may fuel confirmation bias, evidenced by a statistically significant positive linear-mixed-effects model coefficient linking AI recommendations mirroring flawed human judgment and alignment with system advice. Conversely, time pressure appeared to weaken this relationship. These findings highlight potential risks of AI use in healthcare and aim to support the safe integration of clinical decision support systems.

When Two Wrongs Don't Make a Right" -- Examining Confirmation Bias and the Role of Time Pressure During Human-AI Collaboration in Computational Pathology

TL;DR

It is suggested that AI integration may fuel confirmation bias, evidenced by a statistically significant positive linear-mixed-effects model coefficient linking AI recommendations mirroring flawed human judgment and alignment with system advice, and time pressure appeared to weaken this relationship.

Abstract

Artificial intelligence (AI)-based decision support systems hold promise for enhancing diagnostic accuracy and efficiency in computational pathology. However, human-AI collaboration can introduce and amplify cognitive biases, such as confirmation bias caused by false confirmation when erroneous human opinions are reinforced by inaccurate AI output. This bias may worsen when time pressure, ubiquitously present in routine pathology, strains practitioners' cognitive resources. We quantified confirmation bias triggered by AI-induced false confirmation and examined the role of time constraints in a web-based experiment, where trained pathology experts (n=28) estimated tumor cell percentages. Our results suggest that AI integration may fuel confirmation bias, evidenced by a statistically significant positive linear-mixed-effects model coefficient linking AI recommendations mirroring flawed human judgment and alignment with system advice. Conversely, time pressure appeared to weaken this relationship. These findings highlight potential risks of AI use in healthcare and aim to support the safe integration of clinical decision support systems.

Paper Structure

This paper contains 26 sections, 1 equation, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Medical Decision Making and Confirmation Bias
  4. Confirmation Bias in Human-Computer Interaction
  5. The Influence of Time Pressure
  6. Summary of Existing Research and Its Implications for Our Study
  7. Methods
  8. Study Task
  9. Dataset
  10. User Interface Design
  11. Experimental Design
  12. Confirmation bias
  13. Alignment with AI advice
  14. Confidence
  15. Participants
  16. ...and 11 more sections

Figures (3)

  • Figure 1: Study interface as seen by participants during tcp assessment. Depending on the experimental condition, the AI component, including the AI prediction, model reasoning explanations via prototypes and cell detection visualizations, and the reverse countdown timer element were made visible. The arrows illustrate how the interface evolved as the timer elapsed, creating a sense of urgency and discouraging user engagement after countdown expiration.
  • Figure 2: A scatter plot illustrating the relationship between the distance of AI advice from independent estimates and the closeness of AI-aided estimates to AI predictions (based on LMM 1, with a positive slope of approximately 0.61). P-values were derived using the lmerTest library in R with t-tests based on Satterthwaite's method.
  • Figure 3: Bar plot showcasing the difference in coefficients for the predictor variables -- baseline estimate and AI prediction -- between congruent and incongruent AI advice, as derived from LMMs 2 and 3. P-values were derived using the lmerTest library in R with t-tests based on Satterthwaite's method. Clearly, the influence of the AI prediction is elevated in cases where AI advice and the independent estimate are congruent, showcasing confirmation bias. A tabular version of the results shown in this graphic can be found in Table \ref{['tab:weightedavg']} of the appendix.