Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Believing vs. Achieving -- The Disconnect between Efficacy Beliefs and Collaborative Outcomes

Philipp Spitzer, Joshua Holstein

TL;DR

This work investigates how efficacy beliefs translate into instance-wise efficacy judgments under varying contextual information, and reveals efficacy beliefs as persistent cognitive anchors, leading to systematic AI optimism.

Abstract

As artificial intelligence (AI) becomes increasingly integrated into workflows, humans must decide when to rely on AI advice. These decisions depend on general efficacy beliefs, i.e., humans' confidence in their own abilities and their perceptions of AI competence. While prior work has examined factors influencing AI reliance, the role of efficacy beliefs in shaping collaboration remains underexplored. Through a controlled experiment (N=240) where participants made repeated delegation decisions, we investigate how efficacy beliefs translate into instance-wise efficacy judgments under varying contextual information. Our explorative findings reveal efficacy beliefs as persistent cognitive anchors, leading to systematic "AI optimism". Contextual information operates asymmetrically: while AI performance information selectively eliminates the AI optimism bias, data or AI information amplify how efficacy discrepancies influence delegation decisions. Although efficacy discrepancies influence delegation behavior, they show weaker effects on human-AI team performance. As these findings challenge transparency-focused approaches, we propose design guidelines for effective collaborative settings.

Believing vs. Achieving -- The Disconnect between Efficacy Beliefs and Collaborative Outcomes

TL;DR

This work investigates how efficacy beliefs translate into instance-wise efficacy judgments under varying contextual information, and reveals efficacy beliefs as persistent cognitive anchors, leading to systematic AI optimism.

Abstract

As artificial intelligence (AI) becomes increasingly integrated into workflows, humans must decide when to rely on AI advice. These decisions depend on general efficacy beliefs, i.e., humans' confidence in their own abilities and their perceptions of AI competence. While prior work has examined factors influencing AI reliance, the role of efficacy beliefs in shaping collaboration remains underexplored. Through a controlled experiment (N=240) where participants made repeated delegation decisions, we investigate how efficacy beliefs translate into instance-wise efficacy judgments under varying contextual information. Our explorative findings reveal efficacy beliefs as persistent cognitive anchors, leading to systematic "AI optimism". Contextual information operates asymmetrically: while AI performance information selectively eliminates the AI optimism bias, data or AI information amplify how efficacy discrepancies influence delegation decisions. Although efficacy discrepancies influence delegation behavior, they show weaker effects on human-AI team performance. As these findings challenge transparency-focused approaches, we propose design guidelines for effective collaborative settings.
Paper Structure (17 sections, 3 equations, 7 figures, 8 tables)

This paper contains 17 sections, 3 equations, 7 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Concepts
  4. The Role of Contextual Information in Shaping Efficacy Beliefs
  5. Methodology
  6. Experimental Design
  7. Framework for Efficacy-Based Delegation
  8. Results
  9. RQ1 - Relationship of General Efficacy Beliefs and Instance-Wise Efficacy Judgments
  10. RQ2 - Contextual Information on General Efficacy Beliefs and Instance-Wise Judgments
  11. RQ3 - General Efficacy Beliefs and Instance-Wise Judgments on Delegation and Performance
  12. Discussion
  13. Summary of Results
  14. Implications
  15. Limitations and Future Work
  16. ...and 2 more sections

Figures (7)

  • Figure 1: Experimental design examining how contextual information influence general efficacy beliefs and instance-wise efficacy judgments, and how these relationships translate into delegation decisions in AI-assisted decision-making.
  • Figure 2: General efficacy beliefs and instance-wise efficacy judgments for self-efficacy and AI efficacy. AI efficacy discrepancies are larger than for self-efficacy.
  • Figure 3: Instance-wise deviations for self-efficacy and AI efficacy.
  • Figure 4: Interaction effects of the conditions on the relationship of self-efficacy discrepancies and delegation behavior.
  • Figure 5: Interaction effects of the conditions on the relationship of AI efficacy discrepancies and delegation behavior.
  • ...and 2 more figures