The Evaluation Gap in Medicine, AI and LLMs: Navigating Elusive Ground Truth & Uncertainty via a Probabilistic Paradigm

TL;DR

The paper addresses the problem that medical AI benchmarking often ignores ground-truth uncertainty, which can distort apparent performance differences between experts and models. It introduces a probabilistic paradigm where the majority ground-truth label is followed by an expert with probability $p_d$, and derives expected accuracy and F1 via a binomial framework, highlighting the need for uncertainty-stratified evaluation. Through simulations and real-world CheXpert data, the authors show that high absolute performance is attainable mainly in high-certainty bins ($p_d$ near 1), while low-certainty settings can erase meaningful distinctions between humans and AI systems. The work provides practical recommendations to stratify reporting by ground-truth agreement, document annotation counts and Bin-wise performance, and approach evaluation with uncertainty-aware metrics to avoid misleading conclusions. This paradigm has significant implications for how medical AI benchmarks are interpreted and how regulatory and clinical deployment decisions should be informed.

Abstract

Benchmarking the relative capabilities of AI systems, including Large Language Models (LLMs) and Vision Models, typically ignores the impact of uncertainty in the underlying ground truth answers from experts. This ambiguity is particularly consequential in medicine where uncertainty is pervasive. In this paper, we introduce a probabilistic paradigm to theoretically explain how high certainty in ground truth answers is almost always necessary for even an expert to achieve high scores, whereas in datasets with high variation in ground truth answers there may be little difference between a random labeller and an expert. Therefore, ignoring uncertainty in ground truth evaluation data can result in the misleading conclusion that a non-expert has similar performance to that of an expert. Using the probabilistic paradigm, we thus bring forth the concepts of expected accuracy and expected F1 to estimate the score an expert human or system can achieve given ground truth answer variability. Our work leads to the recommendation that when establishing the capability of a system, results should be stratified by probability of the ground truth answer, typically measured by the agreement rate of ground truth experts. Stratification becomes critical when the overall performance drops below a threshold of 80%. Under stratified evaluation, performance comparison becomes more reliable in high certainty bins, mitigating the effect of the key confounding factor -- uncertainty.

Figures (6)

• Figure 1: Simulating the impact of agreement for the majority label (X axis). The Y axis is the performance of a simulated human (H) who follows the majority label distribution ($p_d$) vs. the random labeler (R) performance across different positive label ratios $m \in {0.1, 0.3, 0.5}$. At low certainty, the difference between human labeler distribution and random labeler is relatively low compared to high certainty. Recall and Accuracy are unaffected by the positive label ratio. The vertical bars indicate standard deviation.
• Figure 2: Proportion of samples (CheXpert test set with 500 samples) by agreement level as annotated by 5 radiologists independently. The number of samples with 100% agreement ($p_d=1.0$) on positive findings drops dramatically to less than 0.01 for 4 of 5 the pathologies. The exception is lung opacity, where the proportion of samples approximately remains the same regardless of agreement levels. Agreement level for all pathologies are in Appendix \ref{['fig:chextpertagreement']}.
• Figure 3: Performance of Models (M) vs. human (H) radiologists, compared against ground truth (majority label across 5 human radiologists). H is the average performance of additional 3 radiologists who are different from the 5 radiologists used to determine the ground truth. The data is stratified by $p_d$ probability of observed agreement among the 5 radiologists for the majority label, corresponding positive class ratio is $m$. Dashed lines are theoretical expected performance at given $p_d$ and $m$ according to equations \ref{['equ:expectedacc']} and \ref{['equ:equationf1']}, ($p_d\rightarrow1.0$ is approximated as $p_d=0.985$). Humans are able to achieve a relatively high F1-score ($> 0.8$) as $p_d\rightarrow1.0$, whereas $p_d\rightarrow0.6$ the peak performance drops close to a random labeller baseline. At low positive class ratio ($m<0.01$), even at $p_d\rightarrow1.0$ (see Pneumonia and Consolidation), F1 is $< 0.35$. $\Delta = H - M$ is the vertical distance illustrated by the length of the vertical line connecting the human and the model performance. For Pneumonia, humans have much lower than expected F1 as $p_d\rightarrow0.6$, but it has only 22 samples in that bin. Detailed tabular data is in Appendix Tables \ref{['tab:appendix:gemini_profull']}, \ref{['tab:appendix:gemini25pro']}, and \ref{['tab:gpt5.1']}.
• Figure 4: Distribution of $\Delta=H-M$ in scores using a boxplot across all 3 models, where each $\Delta_{i, o, p_d}=H_{o, p_d}-M_{i, o, p_d}$ where $i\in\{\text{Gemini 3}, \text{Gemini 2.5}, \text{GPT 5.1} \}$, $p_d\in\{0.6,0.8,1.0, \text{All}\}$, $o \in \text{Pathologies}$. $H_{o, p_d}$ is the human score (F1 or accuracy) against the pathology $o$ at $p_d$. Similarly, $M_{i, o, p_d}$ is the performance of the model $i$ against the pathology $o$ at $p_d$. Average $\Delta$ is generally higher as $p_d\rightarrow1.0$ compared to $p_d\rightarrow0.6$ where $\Delta$ F1 is close to zero. Significance t-test: **** ($p<=0.0001$), *** ($p<=0.001$), ** ($p<=0.01$), * ($p<=0.05$), ns (not significant).
• Figure 5: Qualitative examples for model false negatives for Cardiomegaly. 5/5 ($p_d\rightarrow1.0$) ground truth radiologists as well as 2/3 additional radiologists have identified a positive finding.