MedUHIP: Towards Human-In-the-Loop Medical Segmentation

TL;DR

Medical image segmentation suffers from intrinsic boundary ambiguity and inter-clinician variability, hindering reliable quantitative analysis. MedUHIP combines an uncertainty-aware segmentation mechanism with a human-in-the-loop framework, generating multiple plausible segmentations and refining them through clinician feedback to produce a fused soft prediction $P_{soft}^{t}$. The approach introduces a Sampling Net that learns clinician preferences within a Gaussian mixture sampling space, enabling iterative refinement while the Segmentation Net maintains high accuracy. Across REFUGE2, LIDC-IDRI, and QUBIQ, MedUHIP achieves state-of-the-art Dice scores with fewer interactions, demonstrating robust handling of uncertainty and practical potential for clinical adoption.

Abstract

Although segmenting natural images has shown impressive performance, these techniques cannot be directly applied to medical image segmentation. Medical image segmentation is particularly complicated by inherent uncertainties. For instance, the ambiguous boundaries of tissues can lead to diverse but plausible annotations from different clinicians. These uncertainties cause significant discrepancies in clinical interpretations and impact subsequent medical interventions. Therefore, achieving quantitative segmentations from uncertain medical images becomes crucial in clinical practice. To address this, we propose a novel approach that integrates an \textbf{uncertainty-aware model} with \textbf{human-in-the-loop interaction}. The uncertainty-aware model proposes several plausible segmentations to address the uncertainties inherent in medical images, while the human-in-the-loop interaction iteratively modifies the segmentation under clinician supervision. This collaborative model ensures that segmentation is not solely dependent on automated techniques but is also refined through clinician expertise. As a result, our approach represents a significant advancement in the field which enhances the safety of medical image segmentation. It not only offers a comprehensive solution to produce quantitative segmentation from inherent uncertain medical images, but also establishes a synergistic balance between algorithmic precision and clincian knowledge. We evaluated our method on various publicly available multi-clinician annotated datasets: REFUGE2, LIDC-IDRI and QUBIQ. Our method showcases superior segmentation capabilities, outperforming a wide range of deterministic and uncertainty-aware models. We also demonstrated that our model produced significantly better results with fewer interactions compared to previous interactive models. We will release the code to foster further research in this area.

Figures (5)

• Figure 1: A preliminary experiment in testing the impact of different interaction (i.e. clicking) preferences, conducted for the optic cup segmentation on REFUGE2 test set under SAM's structure with Dice score. It indicates that the segmentation performance significantly varies across different interaction strategies, regardless of the number of clicks.
• Figure 2: An overall workflow of our MedUHIP model.
• Figure 3: The architecture of Segmentation Net and Sampling Net.
• Figure 4: Visualisation results produced by determinstic models, uncertainty-based models, interactive models, our MedUHIP method and the ground truth.
• Figure 5: Boxplot of sampling space values based on different clinicians. We draw samples with the mean, variance, weight after six interactions for each clinician.