Auditing Significance, Metric Choice, and Demographic Fairness in Medical AI Challenges

TL;DR

The paper introduces RankInsight, an open-source toolkit designed to address three flaws in medical AI leaderboards: lack of statistical significance testing, inappropriate single-metric rankings, and neglect of demographic fairness. It provides significance maps from pairwise statistical tests, organ-aware metrics (DSC vs NSD) with per-organ rankings and an Overall Score, and demographic fairness auditing with intersectional analysis. Applied to the TotalSegmentator challenge and a Johns Hopkins dataset, the approach shows that small score differences often lack significance, organ-aware metrics can flip rankings, and substantial demographic disparities can exist across subgroups. RankInsight thus enables more robust, clinically meaningful, and fair comparisons across medical AI submissions.

Abstract

Open challenges have become the de facto standard for comparative ranking of medical AI methods. Despite their importance, medical AI leaderboards exhibit three persistent limitations: (1) score gaps are rarely tested for statistical significance, so rank stability is unknown; (2) single averaged metrics are applied to every organ, hiding clinically important boundary errors; (3) performance across intersecting demographics is seldom reported, masking fairness and equity gaps. We introduce RankInsight, an open-source toolkit that seeks to address these limitations. RankInsight (1) computes pair-wise significance maps that show the nnU-Net family outperforms Vision-Language and MONAI submissions with high statistical certainty; (2) recomputes leaderboards with organ-appropriate metrics, reversing the order of the top four models when Dice is replaced by NSD for tubular structures; and (3) audits intersectional fairness, revealing that more than half of the MONAI-based entries have the largest gender-race discrepancy on our proprietary Johns Hopkins Hospital dataset. The RankInsight toolkit is publicly released and can be directly applied to past, ongoing, and future challenges. It enables organizers and participants to publish rankings that are statistically sound, clinically meaningful, and demographically fair.

Figures (4)

• Figure 1: Illustration of calculation of DSC and NSD. DSC (left) is an area overlap-based metric, favoring area agreement. NSD (right) is a perimeter-based metric, calculating the proportion of an object's boundary ($P_Y, P_{\hat{Y}}$) that lies within the other object's frontier region $\pi_Y^\tau$. $\tau$ is the distance threshold that provides leniency for non-perfect boundary matches.
• Figure 2: Pairwise significance map of model rankings on TotalSegmentator. Dark–green cells denote comparisons where the row method is ranked higher than the column method at $p<0.001$. Progressively lighter shades indicate lower, but still meaningful, confidence tiers. nnU-Net-based submissions (dark red) form a clear top tier that consistently outperforms the remaining models (upper-right quadrant), whereas MONAI-based models (gold) appear to underperform (lower right quadrant). Intra-model family comparisons (close to the diagonal) contain many gray cells and light-green cells, cautioning their comparative ranking is less certain.
• Figure 3: Relative ranking of methods based on their per-organ performance, using DSC for blob-like organs and NSD for tubular, elongated organs. Average Scores for each metric, and Overall (unweighted) score are calculated. Table is sorted by Average DSC score. Average NSD Score shows a reversal of the model rankings for the top-performing, nnU-Net-based methods.
• Figure 4: Split Violin plots for least (left) and most (right) equitable models tested, on an Intersectional view of Gender-Race. On each split violin plot, left side is a Kernel Density Estimate of the model's NSD Score on African American Female patients, and right on Asian Male patients. Mean NSD score is denoted by a solid red line. Demographic Parity Difference calculates the difference of means. Three UNETR-based architectures (UNETR, Swin UNETR and Swin UNETR CLIP) and UNEST have the biggest gender-race discrepancy. Two nnU-net-based models (NexToU, MedNeXt) one VL-Based (U-Net CLIP) and SAM-Adapter were most equitable. Most equitable does not imply best, as highlighted by SAM-Adapter.