Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Human-Alignment Influences the Utility of AI-assisted Decision Making

Nina L. Corvelo Benz, Manuel Gomez Rodriguez

TL;DR

The results show a positive association between the degree of alignment and the utility of AI-assisted decision making and post-processing the AI confidence values to achieve multicalibration with respect to the participants’ confidence on their own predictions increases both the degree of alignment and the utility of AI-assisted decision making.

Abstract

Whenever an AI model is used to predict a relevant (binary) outcome in AI-assisted decision making, it is widely agreed that, together with each prediction, the model should provide an AI confidence value. However, it has been unclear why decision makers have often difficulties to develop a good sense on when to trust a prediction using AI confidence values. Very recently, Corvelo Benz and Gomez Rodriguez have argued that, for rational decision makers, the utility of AI-assisted decision making is inherently bounded by the degree of alignment between the AI confidence values and the decision maker's confidence on their own predictions. In this work, we empirically investigate to what extent the degree of alignment actually influences the utility of AI-assisted decision making. To this end, we design and run a large-scale human subject study (n=703) where participants solve a simple decision making task - an online card game - assisted by an AI model with a steerable degree of alignment. Our results show a positive association between the degree of alignment and the utility of AI-assisted decision making. In addition, our results also show that post-processing the AI confidence values to achieve multicalibration with respect to the participants' confidence on their own predictions increases both the degree of alignment and the utility of AI-assisted decision making.

Human-Alignment Influences the Utility of AI-assisted Decision Making

TL;DR

The results show a positive association between the degree of alignment and the utility of AI-assisted decision making and post-processing the AI confidence values to achieve multicalibration with respect to the participants’ confidence on their own predictions increases both the degree of alignment and the utility of AI-assisted decision making.

Abstract

Whenever an AI model is used to predict a relevant (binary) outcome in AI-assisted decision making, it is widely agreed that, together with each prediction, the model should provide an AI confidence value. However, it has been unclear why decision makers have often difficulties to develop a good sense on when to trust a prediction using AI confidence values. Very recently, Corvelo Benz and Gomez Rodriguez have argued that, for rational decision makers, the utility of AI-assisted decision making is inherently bounded by the degree of alignment between the AI confidence values and the decision maker's confidence on their own predictions. In this work, we empirically investigate to what extent the degree of alignment actually influences the utility of AI-assisted decision making. To this end, we design and run a large-scale human subject study (n=703) where participants solve a simple decision making task - an online card game - assisted by an AI model with a steerable degree of alignment. Our results show a positive association between the degree of alignment and the utility of AI-assisted decision making. In addition, our results also show that post-processing the AI confidence values to achieve multicalibration with respect to the participants' confidence on their own predictions increases both the degree of alignment and the utility of AI-assisted decision making.
Paper Structure (2 sections, 10 equations, 17 figures, 2 tables)

This paper contains 2 sections, 10 equations, 17 figures, 2 tables.

Table of Contents

  1. Additional Experimental Results
  2. Expected Calibration Error

Figures (17)

  • Figure 1: Our simple AI-assisted decision making task.
  • Figure 2: AI confidence $a$ and fraction of red cards shown $z$ against the fraction of red cards $r$ per game played by participants in each group. Games in which the participant are most likely mislead since the majority of cards shown have the opposite of the overall majority color are highlighted in gray.
  • Figure 3: Heatmap of decision probabilities of guessing red stratified by participants' initial confidence and AI confidence shown, and averaged over participants. The initial confidence recorded is discretized into four bins---very low, low, high, very high---denoting the confidence of the participants that the color of the picked card will be red. Bins with 10 or less data points are not displayed.
  • Figure 4: Results for groups $\color{boxgray}\textcolor{bluearrow}{\leftarrow} \textcolor{circlegray}{\bullet}\,\textcolor{circlegray}{\bullet}\textcolor{redarrow}{\rightarrow}\ $, $\color{boxgray} \textcolor{circlegray}{\bullet}\quad\textcolor{circlegray}{\bullet}\ $ and $\color{boxgray}\textcolor{circlegray}{\bullet}\textcolor{redarrow}{\rightarrow} \textcolor{bluearrow}{\leftarrow}\textcolor{circlegray}{\bullet}\ $.
  • Figure 5: Results for groups R and $\color{boxgray}\textcolor{circlegray}{\bullet}\textcolor{redarrow}{\rightarrow} \textcolor{bluearrow}{\leftarrow}\textcolor{circlegray}{\bullet}\ $.
  • ...and 12 more figures