Resource-efficient Automatic Refinement of Segmentations via Weak Supervision from Light Feedback

TL;DR

The paper addresses the need to refine automatic segmentations from foundation models to clinical accuracy without requiring dense GT annotations. It introduces SCORE, a weakly supervised refinement framework that uses region-wise quality scores and error-type labels to drive a morphology-inspired loss guiding corrections to under- or over-segmented regions. The refinement network uses a 3-channel input (image, initial segmentation, boundary prior) and is implemented as a 3D UNet, trained with data augmentation. On humerus CT, SCORE improves TotalSegmentator predictions to perform comparably to state-of-the-art semi- and fully supervised refinements, while reducing annotation burden by about 95%, demonstrating practical potential for clinical deployment.

Abstract

Delineating anatomical regions is a key task in medical image analysis. Manual segmentation achieves high accuracy but is labor-intensive and prone to variability, thus prompting the development of automated approaches. Recently, a breadth of foundation models has enabled automated segmentations across diverse anatomies and imaging modalities, but these may not always meet the clinical accuracy standards. While segmentation refinement strategies can improve performance, current methods depend on heavy user interactions or require fully supervised segmentations for training. Here, we present SCORE (Segmentation COrrection from Regional Evaluations), a weakly supervised framework that learns to refine mask predictions only using light feedback during training. Specifically, instead of relying on dense training image annotations, SCORE introduces a novel loss that leverages region-wise quality scores and over/under-segmentation error labels. We demonstrate SCORE on humerus CT scans, where it considerably improves initial predictions from TotalSegmentator, and achieves performance on par with existing refinement methods, while greatly reducing their supervision requirements and annotation time. Our code is available at: https://gitlab.inria.fr/adelangl/SCORE.

Figures (2)

• Figure 1: Overview of SCORE, our weakly supervised framework to refine segmentation from an external tool using only light feedback. The network takes as input a 3D image, its initial segmentation, and a probability map for additional edge priors.
• Figure 2: Example segmentation for all methods on a CHU-Full subject, with associated correction time and interaction type.