As Confidence Aligns: Exploring the Effect of AI Confidence on Human Self-confidence in Human-AI Decision Making

TL;DR

<3-5 sentence high-level summary>: The paper investigates how AI confidence influences human self-confidence in human-AI decision making and whether this alignment persists after AI involvement. Using a randomized online experiment with income-prediction tasks and three collaboration paradigms, the study demonstrates that human self-confidence tends to align with AI confidence during joint tasks and that this alignment can persist in subsequent independent tasks, though real-time feedback can dampen the effect. The alignment affects human self-confidence calibration, often impairing calibration and reducing decision-making efficacy in some conditions, while potentially improving calibration for others depending on relative confidences. The work provides theoretical expansion of confidence alignment into human-AI contexts and offers design guidance for mitigating miscalibration and enhancing complementary collaboration in practical systems.

Abstract

Complementary collaboration between humans and AI is essential for human-AI decision making. One feasible approach to achieving it involves accounting for the calibrated confidence levels of both AI and users. However, this process would likely be made more difficult by the fact that AI confidence may influence users' self-confidence and its calibration. To explore these dynamics, we conducted a randomized behavioral experiment. Our results indicate that in human-AI decision-making, users' self-confidence aligns with AI confidence and such alignment can persist even after AI ceases to be involved. This alignment then affects users' self-confidence calibration. We also found the presence of real-time correctness feedback of decisions reduced the degree of alignment. These findings suggest that users' self-confidence is not independent of AI confidence, which practitioners aiming to achieve better human-AI collaboration need to be aware of. We call for research focusing on the alignment of human cognition and behavior with AI.

Figures (8)

• Figure 1: Diagrams of three human-AI decision making paradigms. (a): AI as advisor. (b): AI as peer collaborator. (c): AI as decision-maker under human supervision.
• Figure 2: Flowchart of the experimental procedure.
• Figure 3: Interface of the income prediction task (an example on a task instance from stage 2 and under the paradigm where AI acts as a peer collaborator). A: The profile table of the income prediction task including 8 attributes. B: AI prediction and confidence level about the task. They are only presented in stage 2, after users report its prediction and self-confidence at first. C: Users' prediction and self-confidence level about the task. At the beginning of each task from each stage, users need to report and submit their decision and self-confidence here at first. D: The final decision and confidence about it, only applying to stage 2. For the paradigm where AI acts as an advisor, users need to make the final decision here. For the other two paradigms, the system would make the final decision according to the rules and present the result here. E: Real-time correctness feedback for users' decision (stage 1, 2 and 3), AI's decision (stage 2), and the joint final decision (stage 2): under conditions with real-time feedback, it would be displayed after users submit their own decisions in stages 1 and 3, and after the final decision is made in stage 2.
• Figure 4: Absolute confidence difference between participants' self-confidence and AI confidence (80.40%). The smaller the difference, the higher the alignment of participants' self-confidence with AI confidence. The mean values and standard deviations of the absolute confidence difference in each stage are shown in the tables below corresponding subfigures. The significance levels are labeled ($p<0.05$: *, $p<0.01$: **, $p<0.001$: ***). (a): The jitter plot on the left displays the absolute confidence difference for each participant from stage 1 to stage 3, with results from the same participants connected by lines. The box plot on the right illustrates the distribution of the absolute confidence differences at each stage, where the center line represents the median, the box boundaries are the upper and lower quartiles, and the whiskers extend to the extreme points, with outliers also marked. (b): Line plot about the absolute confidence difference across three stages and two different real-time conditions. The points represent mean values, and the error bars represent one standard error.
• Figure 5: Sliding window average (k=4) of the decision-by-decision absolute confidence difference between participants' self-confidence and AI's mean confidence over 120 decision tasks in this experiment with 95% confidence interval displayed. The absolute confidence difference reduced at stage 2, where participants were given AI confidence levels.