MelanomaNet: Explainable Deep Learning for Skin Lesion Classification

TL;DR

MelanomaNet addresses the interpretability gap in skin lesion classification by integrating an EfficientNet V2-M backbone with four interpretability channels: GradCAM++ attention, automated ABCDE criterion analysis, FastCAV concept explanations, and Monte Carlo Dropout uncertainty quantification. The system demonstrates strong classification performance on the ISIC 2019 dataset while providing clinically meaningful explanations and uncertainty signals to support human review. Key contributions include a multi-modal explainability framework, alignment metrics validating attention to dermatologically relevant features, and a decomposed uncertainty strategy to flag unreliable predictions. Collectively, the work suggests that explainability and clinical utility can be pursued in tandem to enhance trust and adoption in dermatology workflows.

Abstract

Automated skin lesion classification using deep learning has shown remarkable accuracy, yet clinical adoption remains limited due to the "black box" nature of these models. We present MelanomaNet, an explainable deep learning system for multi-class skin lesion classification that addresses this gap through four complementary interpretability mechanisms. Our approach combines an EfficientNet V2 backbone with GradCAM++ attention visualization, automated ABCDE clinical criterion extraction, Fast Concept Activation Vectors (FastCAV) for concept-based explanations, and Monte Carlo Dropout uncertainty quantification. We evaluate our system on the ISIC 2019 dataset containing 25,331 dermoscopic images across 9 diagnostic categories. Our model achieves 85.61% accuracy with a weighted F1 score of 0.8564, while providing clinically meaningful explanations that align model attention with established dermatological assessment criteria. The uncertainty quantification module decomposes prediction confidence into epistemic and aleatoric components, enabling automatic flagging of unreliable predictions for clinical review. Our results demonstrate that high classification performance can be achieved alongside comprehensive interpretability, potentially facilitating greater trust and adoption in clinical dermatology workflows. The source code is available at https://github.com/suxrobgm/explainable-melanoma

Figures (3)

• Figure 1: MelanomaNet system architecture. The main classification pipeline (top) processes dermoscopic images through EfficientNet V2-M to produce predictions across 8 classes. Four explainability modules (bottom, dashed box) provide complementary interpretations: ABCDE clinical criteria from image segmentation, GradCAM++ attention from feature maps, FastCAV concept scores from extracted features, and MC Dropout uncertainty estimates from stochastic forward passes. The GradCAM-ABCDE alignment module validates correspondence between model attention and clinical features.
• Figure 2: Analysis of a benign nevus (NV) with medium ABCDE risk. The model classifies with 94.49% confidence. Top row: original image, GradCAM++ heatmap, and overlay with prediction. Middle row: ABCDE criterion visualizations showing asymmetry axes, border contour, color palette, and diameter measurement. Bottom panels: uncertainty decomposition (left) and FastCAV concept importance scores (right).
• Figure 3: Analysis of a melanoma (MEL) with medium ABCDE risk. The model classifies with 100% confidence but the uncertainty module flags this as "UNCERTAIN" due to high aleatoric uncertainty (0.75). ABCDE criteria show acceptable asymmetry (0.12) and borders (0.26), but flag multiple colors (6) and large diameter (409px). FastCAV analysis reveals large diameter strongly supports the prediction (+5.32) while asymmetry opposes it (-2.77).