Hybrid Interpretable Deep Learning Framework for Skin Cancer Diagnosis: Integrating Radial Basis Function Networks with Explainable AI

TL;DR

This work tackles the interpretability gap in skin cancer diagnosis by proposing a hybrid framework that couples CNN-based feature extraction with a Radial Basis Function Network (RBF-NN) built on automatically selected prototypes. By segmenting images, extracting embeddings, and grounding decisions in localized prototypes through an RBF layer, the model provides faithful, interpretable explanations aligned with clinical reasoning. Evaluations on ISIC-2016 and ISIC-2017 demonstrate competitive accuracy, particularly with ResNet-50 features, while maintaining transparency through prototype-based reasoning and active-learning-driven prototype selection. The approach aims to bridge prediction performance and clinician trust, offering a scalable path for explainable AI in high-stakes medical imaging tasks, with potential applicability beyond dermatology. Future work includes addressing dataset size and computational demands to enhance robustness and deployment in resource-constrained settings.

Abstract

Skin cancer is one of the most prevalent and potentially life-threatening diseases worldwide, necessitating early and accurate diagnosis to improve patient outcomes. Conventional diagnostic methods, reliant on clinical expertise and histopathological analysis, are often time-intensive, subjective, and prone to variability. To address these limitations, we propose a novel hybrid deep learning framework that integrates convolutional neural networks (CNNs) with Radial Basis Function (RBF) Networks to achieve high classification accuracy and enhanced interpretability. The motivation for incorporating RBF Networks lies in their intrinsic interpretability and localized response to input features, which make them well-suited for tasks requiring transparency and fine-grained decision-making. Unlike traditional deep learning models that rely on global feature representations, RBF Networks allow for mapping segments of images to chosen prototypes, exploiting salient features within a single image. This enables clinicians to trace predictions to specific, interpretable patterns. The framework incorporates segmentation-based feature extraction, active learning for prototype selection, and K-Medoids clustering to focus on these salient features. Evaluations on the ISIC 2016 and ISIC 2017 datasets demonstrate the model's effectiveness, achieving classification accuracies of 83.02\% and 72.15\% using ResNet50, respectively, and outperforming VGG16-based configurations. By generating interpretable explanations for predictions, the framework aligns with clinical workflows, bridging the gap between predictive performance and trustworthiness. This study highlights the potential of hybrid models to deliver actionable insights, advancing the development of reliable AI-assisted diagnostic tools for high-stakes medical applications.

Figures (6)

• Figure 1: Architecture of proposed Radial basis neural network
• Figure 2: Images from ISIC 2016 of both classes
• Figure 3: Images from ISIC 2017 of all classes
• Figure 4: Segments from ISIC 2016 chosen as prototypes by the model from each class
• Figure 5: Segments from ISIC 2017 chosen as prototypes by the model from each class