Confidence is detection-like in high-dimensional spaces

TL;DR

It is shown that Bayesian confidence estimates also exhibit heightened sensitivity to decision-congruent evidence in higher-dimensional signal detection theoretic spaces, leading to detection-like confidence criteria.

Abstract

Confidence estimates are often "detection-like" - driven by positive evidence in favour of a decision. This empirical observation has been interpreted as showing human metacognition is limited by biases or heuristics. Here we show that Bayesian confidence estimates also exhibit heightened sensitivity to decision-congruent evidence in higher-dimensional signal detection theoretic spaces, leading to detection-like confidence criteria. This effect is due to a nonlinearity induced by normalisation of confidence by a large number of unchosen alternatives. Our analysis suggests that detection-like confidence is rational when computing confidence in a higher-dimensional evidence space than that assumed by the experimenter.

Figures (16)

• Figure 1: Relationship between sensory evidence and confidence for (A) discrimination tasks and (B) detection tasks in a 2-dimensional signal detection theory (SDT) model. Circles indicate bivariate Gaussian evidence distributions produced by stimulus S1 (blue), stimulus S2 (yellow), and an absent stimulus (grey). Shaded areas indicate the optimal confidence ratings for an ideal Bayesian observer. (C-D) Two heuristic strategies for responding in discrimination tasks. Under the balance of evidence heuristic (C), confidence is sensitive to relative evidence (the distance from the diagonal). Under the response congruent evidence heuristic (D), confidence is only sensitive to evidence for the chosen option (the distance along the horizontal or vertical axis). Although the balance of evidence heuristic is the optimal strategy for discrimination tasks, confidence ratings tend to resemble the response congruent evidence heuristic, which approximates the optimal strategy in detection tasks. Modified from Mazor2023.
• Figure 2: A-E) Impact of computing confidence in higher-dimensional SDT spaces for a 2D decision between $s_1$ and $s_2$ on (A) confidence, (B) confidence's sensitivity to evidence for the chosen alternative, (C) confidence's sensitivity to evidence for the unchosen alternative, and (D-E) the positive evidence bias (PEB), here computed as the log ratio of the two previous quantities. Confidence is increasingly detection-like (cf. Figure \ref{['fig1']}) as dimensionality increases, exhibiting increased sensitivity to evidence for the chosen alternative relative to the unchosen alternative. F) Regression coefficients relating chosen and unchosen evidence samples to accuracy (left panel) and confidence (right panel). G) Posterior probabilities of each stimulus class (colored lines) and confidence (black line) along the slice indicated by the dashed white line in panel (A) for $k=10$.
• Figure 3: A) Samples from two equal variance stimulus distributions ($s_1$ and $s_2$), colour-coded by the level of confidence attached to a decision arising from each evidence sample, for increasing dimensionality. For a decision of $s_1$, confidence increases from blue to green. For a decision of $s_2$, confidence increases from red to yellow. B, C) Target:non-target variance ratio computed in (B) evidence space and (C) confidence space.
• Figure 4: Impact of computing confidence in higher-dimensional SDT spaces on confidence surfaces for a 3-way decision between $s_1$, $s_2$ and $s_3$. Conventions as in Figure \ref{['fig2']}.
• Figure 5: A) Model architecture Webb2023. An image $x$, belonging to class $y$, was passed through a deep neural network (DNN) encoder $f$, followed by two output heads: $g_{class}$ generated a decision classifying the image, and $g_{conf}$ generated a confidence score by predicting $p(\hat{y} = y)$, the probability that the decision was correct. B, C) Impact of varying the dimensionality of the training set on a positive evidence bias in confidence obtained for MNIST digit classification. The fitted lines are derived from a logarithmic model in which the impact of dimensionality on PEB saturates at higher dimensionalities.