Imbalanced Medical Image Segmentation with Pixel-dependent Noisy Labels

TL;DR

This work tackles pixel-wise noisy labels in medical image segmentation under class imbalance by introducing CLCS, a collaborative two-branch framework guided by a curriculum-based selection and a robust learning objective. The Curriculum Noisy Label Sample Selection (CNS) module uses a discrepancy-driven, two-branch network with a dynamic, class-aware threshold to identify clean labels, while the Noise Balance Loss (NBL) reuses noisy samples with a mixed CE and RCE loss weighted by confidence to improve data utilization. Key contributions include a discrepancy loss to prevent branch collapse, a convex-curvature curriculum threshold to address imbalance, and a balanced loss that mitigates overfitting to noisy labels, validated by extensive experiments on Endovis18 and RIGA showing consistent improvements over robust-loss and pixel-denoising baselines. The approach demonstrates practical impact by enhancing segmentation performance in real-world medical datasets with realistic noise patterns, offering a scalable solution for robust training when clean pixel-level annotations are scarce.

Abstract

Accurate medical image segmentation is often hindered by noisy labels in training data, due to the challenges of annotating medical images. Prior research works addressing noisy labels tend to make class-dependent assumptions, overlooking the pixel-dependent nature of most noisy labels. Furthermore, existing methods typically apply fixed thresholds to filter out noisy labels, risking the removal of minority classes and consequently degrading segmentation performance. To bridge these gaps, our proposed framework, Collaborative Learning with Curriculum Selection (CLCS), addresses pixel-dependent noisy labels with class imbalance. CLCS advances the existing works by i) treating noisy labels as pixel-dependent and addressing them through a collaborative learning framework, and ii) employing a curriculum dynamic thresholding approach adapting to model learning progress to select clean data samples to mitigate the class imbalance issue, and iii) applying a noise balance loss to noisy data samples to improve data utilization instead of discarding them outright. Specifically, our CLCS contains two modules: Curriculum Noisy Label Sample Selection (CNS) and Noise Balance Loss (NBL). In the CNS module, we designed a two-branch network with discrepancy loss for collaborative learning so that different feature representations of the same instance could be extracted from distinct views and used to vote the class probabilities of pixels. Besides, a curriculum dynamic threshold is adopted to select clean-label samples through probability voting. In the NBL module, instead of directly dropping the suspiciously noisy labels, we further adopt a robust loss to leverage such instances to boost the performance.

Figures (10)

• Figure 1: Overview of Collaborative Learning with Curriculum Selection (CLCS). An input image is processed by each of the two network branches individually to generate predictions from distinct viewpoints, facilitated by a discrepancy loss. Leveraging the predictions from the two branches and the original label, the model groups the pixels into a clean set and a noise set by the Curriculum Noisy Label Sample Selection (CNS) module. The network predictions and original labels are integrated through a curriculum dynamic threshold with a robust voting strategy. The model is trained by minimizing the supervised cross-entropy loss for the clean set and the Noise Balance Loss (NBL) for the noise set. The blue block in the BCL module is the mapping layer.
• Figure 2: Illustration of Curriculum Dynamic Threshold (CDT). The $M$ represents a convex function.
• Figure 3: Visual comparison of different noisy labels. Column 1: original images; Column 2: SFDA-Noise labels; Column 3: Noise annotations from Rater 6 (fig. c) and SFDA+ED-Noise labels (fig. g); Column 4: clean segmentation labels.
• Figure 4: Visual comparison with SFDA-Noise on Endovis18. [Dice/mIoU] are given.
• Figure 5: Visual comparison of the segmentation results from different methods. The segmentation results with Real-Noise on the training dataset. The symbol [. / .] denotes [Dice / mIoU] scores.