Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

When Thinking Pays Off: Incentive Alignment for Human-AI Collaboration

Joshua Holstein, Patrick Hemmer, Gerhard Satzger, Wei Sun

TL;DR

This work demonstrates that overreliance on AI in decision making stems from misaligned incentives, not merely cognitive biases. The authors formalize a model showing that effort costs bias humans toward following AI advice and propose a dynamic bonus mechanism to reward independent problem solving. A behavioral experiment with 180 participants using a DenseNet-161 classifier on distorted ImageNet-16H data shows that dynamic bonuses reduce reliance and improve human-AI team accuracy, while static bonuses can degrade performance and induce gaming. The results stress the need for context-sensitive incentive design to realize true human-AI complementarity, with cognitive load partly mediating the effects of dynamic incentives, and point to mechanism design as a key tool for robust collaboration across domains.

Abstract

Collaboration with artificial intelligence (AI) has improved human decision-making across various domains by leveraging the complementary capabilities of humans and AI. Yet, humans systematically overrely on AI advice, even when their independent judgment would yield superior outcomes, fundamentally undermining the potential of human-AI complementarity. Building on prior work, we identify prevailing incentive structures in human-AI decision-making as a structural driver of this overreliance. To address this misalignment, we propose an alternative incentive mechanism designed to counteract systemic overreliance. We empirically evaluate this approach through a behavioral experiment with 180 participants, finding that the proposed mechanism significantly reduces overreliance. We also show that while appropriately designed incentives can enhance collaboration and decision quality, poorly designed incentives may distort behavior, introduce unintended consequences, and ultimately degrade performance. These findings underscore the importance of aligning incentives with task context and human-AI complementarities, and suggest that effective collaboration requires a shift toward context-sensitive incentive design.

When Thinking Pays Off: Incentive Alignment for Human-AI Collaboration

TL;DR

This work demonstrates that overreliance on AI in decision making stems from misaligned incentives, not merely cognitive biases. The authors formalize a model showing that effort costs bias humans toward following AI advice and propose a dynamic bonus mechanism to reward independent problem solving. A behavioral experiment with 180 participants using a DenseNet-161 classifier on distorted ImageNet-16H data shows that dynamic bonuses reduce reliance and improve human-AI team accuracy, while static bonuses can degrade performance and induce gaming. The results stress the need for context-sensitive incentive design to realize true human-AI complementarity, with cognitive load partly mediating the effects of dynamic incentives, and point to mechanism design as a key tool for robust collaboration across domains.

Abstract

Collaboration with artificial intelligence (AI) has improved human decision-making across various domains by leveraging the complementary capabilities of humans and AI. Yet, humans systematically overrely on AI advice, even when their independent judgment would yield superior outcomes, fundamentally undermining the potential of human-AI complementarity. Building on prior work, we identify prevailing incentive structures in human-AI decision-making as a structural driver of this overreliance. To address this misalignment, we propose an alternative incentive mechanism designed to counteract systemic overreliance. We empirically evaluate this approach through a behavioral experiment with 180 participants, finding that the proposed mechanism significantly reduces overreliance. We also show that while appropriately designed incentives can enhance collaboration and decision quality, poorly designed incentives may distort behavior, introduce unintended consequences, and ultimately degrade performance. These findings underscore the importance of aligning incentives with task context and human-AI complementarities, and suggest that effective collaboration requires a shift toward context-sensitive incentive design.

Paper Structure

This paper contains 14 sections, 16 equations, 5 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Description
  4. Effort-Induced Overreliance
  5. Incentive Alignment via Bonus Mechanism
  6. Behavioral Experiment Design
  7. Experimental Results
  8. Task Completion
  9. Human-AI Team Performance
  10. Reliance Behavior
  11. Discussion
  12. Key Findings
  13. Implications
  14. Conclusion

Figures (5)

  • Figure 1: Overview of the 30-instance selection scheme.
  • Figure 2: Distribution of processed instances across experimental conditions.
  • Figure 3: Weighted average human-AI team performance across treatments with 95% confidence intervals. Reference lines show AI-only performance (purple) and human-only performance (dark green) derived from dataset ground truth labels.
  • Figure 4: Weighted average participant reliance across treatments with 95% confidence intervals.
  • Figure 5: Distribution of instances with confidences to complementary subsets.