Generalizing Abstention for Noise-Robust Learning in Medical Image Segmentation

TL;DR

The paper tackles label noise in medical image segmentation by generalizing the abstention mechanism to segmentation, introducing a universal framework that can wrap any base loss ${\mathcal{L}}_X$ to form ${\mathcal{L}}_{abstention}=(1-p_{k+1}){\mathcal{L}}_X+\alpha\left|\log\frac{1-\tilde{\eta}}{1-p_{k+1}}\right|$, guided by an informed regularization term and a power-law auto-tuning of $\alpha$. It presents three loss variants—GAC, SAC, and ADS—that couple abstention with Generalized Cross Entropy, Symmetric Cross Entropy, and the Dice loss, respectively, with ADS adding class-wise abstention capabilities. Evaluations on CaDIS and DSAD under varying noise levels show that abstaining losses consistently outperform non-abstaining baselines, with ADS achieving the strongest gains at higher noise levels and flatter degradation curves overall. The work demonstrates the practical value of selective abstention for reliable medical image segmentation and provides an open-source implementation to facilitate adoption. The proposed modular, loss-agnostic framework enables robust learning from imperfect annotations and paves the way for future work on adaptive noise estimation and real-world annotation variability.

Abstract

Label noise is a critical problem in medical image segmentation, often arising from the inherent difficulty of manual annotation. Models trained on noisy data are prone to overfitting, which degrades their generalization performance. While a number of methods and strategies have been proposed to mitigate noisy labels in the segmentation domain, this area remains largely under-explored. The abstention mechanism has proven effective in classification tasks by enhancing the capabilities of Cross Entropy, yet its potential in segmentation remains unverified. In this paper, we address this gap by introducing a universal and modular abstention framework capable of enhancing the noise-robustness of a diverse range of loss functions. Our framework improves upon prior work with two key components: an informed regularization term to guide abstention behaviour, and a more flexible power-law-based auto-tuning algorithm for the abstention penalty. We demonstrate the framework's versatility by systematically integrating it with three distinct loss functions to create three novel, noise-robust variants: GAC, SAC, and ADS. Experiments on the CaDIS and DSAD medical datasets show our methods consistently and significantly outperform their non-abstaining baselines, especially under high noise levels. This work establishes that enabling models to selectively ignore corrupted samples is a powerful and generalizable strategy for building more reliable segmentation models. Our code is publicly available at https://github.com/wemous/abstention-for-segmentation.

Figures (7)

• Figure 1: The impact of our noise-robust abstention framework. On a CaDIS sample with 25% label noise, the baseline Dice Loss (b) produces a noisy and inaccurate mask. In contrast, our proposed Abstaining Dice Segmenter (ADS) (c) yields a result that is visually cleaner and adheres more closely to the ground truth (a).
• Figure 2: Transforming the output layer from standard pixel-wise abstention (a) to our proposed class-wise abstention head for ADS (b). The dimensions $b, c, h, w$ represent batch size, number of classes, height, and width, respectively. For the Adaptive Average Pool layer, $w, w$ is the output size
• Figure 3: Quantitative comparison of noise-robustness. The plots show the average test mIoU (%) degradation as label noise $\eta$ increases. The flatter curves of the abstaining variants (solid lines) demonstrate their superior resilience compared to non-abstaining baselines (dashed lines). Our proposed losses (GAC, SAC, ADS) are in bold.
• Figure 4: Qualitative comparison on a CaDIS sample at 25% noise. Our proposed abstaining losses (GAC, SAC, ADS) produce masks with higher fidelity and fewer artifacts than their baselines. Abstaining losses are in bold.
• Figure 5: Qualitative comparison on a challenging DSAD sample at 15% noise. Abstaining variants (in bold) yield masks with better structural coherence and fewer spurious activations than their baselines.