Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Uncertainty Quantified Deep Learning and Regression Analysis Framework for Image Segmentation of Skin Cancer Lesions

Elhoucine Elfatimi, Pratik Shah

TL;DR

This work tackles the challenge of trustworthy skin lesion segmentation by introducing pixel-level uncertainty estimation using Monte Carlo dropout and Bayes-by-backprop on deep learning models trained from scratch and with transfer learning. It combines segmentation outputs with region-specific uncertainty maps, and presents Algorithm 1 to predict Dice scores from ROI uncertainties through four linear regression models, validated on ISIC-2019 data. The results show significant correlations (p < 0.05) between ROI uncertainties and Dice performance across nevus, melanoma, and seborrheic keratosis classes, with qualitative analyses indicating that Monte Carlo dropout generally yields stronger segmentation and informative uncertainty patterns. The proposed framework enables uncertainty-aware predictions and offline performance estimation, supporting clinical interpretation and potential OoD adaptation, with code and models publicly available for reproducibility and extension to other dermoscopic datasets.

Abstract

Deep learning models (DLMs) frequently achieve accurate segmentation and classification of tumors from medical images. However, DLMs lacking feedback on their image segmentation mechanisms, such as Dice coefficients and confidence in their performance, face challenges when processing previously unseen images in real-world clinical settings. Uncertainty estimates to identify DLM predictions at the cellular or single-pixel level that require clinician review can enhance trust. However, their deployment requires significant computational resources. This study reports two DLMs, one trained from scratch and another based on transfer learning, with Monte Carlo dropout or Bayes-by-backprop uncertainty estimations to segment lesions from the publicly available The International Skin Imaging Collaboration-19 dermoscopy image database with cancerous lesions. A novel approach to compute pixel-by-pixel uncertainty estimations of DLM segmentation performance in multiple clinical regions from a single dermoscopy image with corresponding Dice scores is reported for the first time. Image-level uncertainty maps demonstrated correspondence between imperfect DLM segmentation and high uncertainty levels in specific skin tissue regions, with or without lesions. Four new linear regression models that can predict the Dice performance of DLM segmentation using constants and uncertainty measures, either individually or in combination from lesions, tissue structures, and non-tissue pixel regions critical for clinical diagnosis and prognostication in skin images (Spearman's correlation, p < 0.05), are reported for the first time for low-compute uncertainty estimation workflows.

Uncertainty Quantified Deep Learning and Regression Analysis Framework for Image Segmentation of Skin Cancer Lesions

TL;DR

This work tackles the challenge of trustworthy skin lesion segmentation by introducing pixel-level uncertainty estimation using Monte Carlo dropout and Bayes-by-backprop on deep learning models trained from scratch and with transfer learning. It combines segmentation outputs with region-specific uncertainty maps, and presents Algorithm 1 to predict Dice scores from ROI uncertainties through four linear regression models, validated on ISIC-2019 data. The results show significant correlations (p < 0.05) between ROI uncertainties and Dice performance across nevus, melanoma, and seborrheic keratosis classes, with qualitative analyses indicating that Monte Carlo dropout generally yields stronger segmentation and informative uncertainty patterns. The proposed framework enables uncertainty-aware predictions and offline performance estimation, supporting clinical interpretation and potential OoD adaptation, with code and models publicly available for reproducibility and extension to other dermoscopic datasets.

Abstract

Deep learning models (DLMs) frequently achieve accurate segmentation and classification of tumors from medical images. However, DLMs lacking feedback on their image segmentation mechanisms, such as Dice coefficients and confidence in their performance, face challenges when processing previously unseen images in real-world clinical settings. Uncertainty estimates to identify DLM predictions at the cellular or single-pixel level that require clinician review can enhance trust. However, their deployment requires significant computational resources. This study reports two DLMs, one trained from scratch and another based on transfer learning, with Monte Carlo dropout or Bayes-by-backprop uncertainty estimations to segment lesions from the publicly available The International Skin Imaging Collaboration-19 dermoscopy image database with cancerous lesions. A novel approach to compute pixel-by-pixel uncertainty estimations of DLM segmentation performance in multiple clinical regions from a single dermoscopy image with corresponding Dice scores is reported for the first time. Image-level uncertainty maps demonstrated correspondence between imperfect DLM segmentation and high uncertainty levels in specific skin tissue regions, with or without lesions. Four new linear regression models that can predict the Dice performance of DLM segmentation using constants and uncertainty measures, either individually or in combination from lesions, tissue structures, and non-tissue pixel regions critical for clinical diagnosis and prognostication in skin images (Spearman's correlation, p < 0.05), are reported for the first time for low-compute uncertainty estimation workflows.
Paper Structure (21 sections, 7 equations, 4 figures, 3 tables, 1 algorithm)

This paper contains 21 sections, 7 equations, 4 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Summary of Contributions
  4. Data Description
  5. Methods
  6. Deep Learning Model Training
  7. Deep Learning Model Prediction And Evaluation
  8. Modelling Performance Based on Uncertainty
  9. Results
  10. Deep Learning Model Performance and Uncertainty Analysis
  11. Performance On Prediction Of Nevus Skin Tissue
  12. Performance On Prediction Of Melanoma Skin Lesions
  13. Performance On Prediction Of Seborrhoeic Keratosis Skin Tissue
  14. Predictive capabilities of the proposed linear models
  15. Comparative Qualitative Evaluation of Deep Learning Models
  16. ...and 6 more sections

Figures (4)

  • Figure 1: Schematic diagram of the deep learning framework for interpretable skin lesion image segmentation with uncertainty quantification and linear regression
  • Figure 2: Segmentation and uncertainty maps obtained from Bayes-by-backrop (Bc) and Monte Carlo dropout (MCD) augmented transfer learning (TL) and trained from scratch (TFS) deep learning models. GT - Ground Truth labels, Pred - Model prediction, Unc-Uncertainty.
  • Figure 3: Visualization of skin cancer segmentation and uncertainty by a Monte Carlo dropout (MCD) model trained from scratch on dermoscopy images. Columns left to right:A Input RGB image; B Clinical ground-truth binary mask; C MCD segmentation binary mask; D MCD uncertainty map for segmentation; E MCD uncertainty map for lesion regions; F MCD uncertainty map for non-lesion regions. The color bar shows model uncertainty, with red indicating high uncertainty and blue indicating low uncertainty (0). 'GT' stands for clinical ground-truth.
  • Figure 4: Visualization of skin cancer segmentation and uncertainty by a Monte Carlo dropout (MCD) model trained with transfer learning on dermoscopy images. Left to right columns:A Input RGB image; B Clinical ground-truth binary mask; C MCD segmentation binary mask; D MCD uncertainty map for segmentation; E MCD uncertainty map for lesion regions; F MCD uncertainty map for non-lesion regions. The color bar shows model uncertainty, with red indicating high uncertainty and blue indicating low uncertainty (0). 'GT' stands for clinical ground-truth.