Improving Prototypical Parts Abstraction for Case-Based Reasoning Explanations Designed for the Kidney Stone Type Recognition

TL;DR

This work tackles in-vivo kidney stone type recognition with an explainable DL model based on prototypical parts (PPs). By introducing a Deep Metric Learning–driven loss (ICNN) and combining it with CE and PP-based objectives (CIC, PPIC), the approach promotes diverse, representative PP prototypes while preserving or improving classification accuracy. Descriptors, including local and global perturbation-based feature analyses, enable interpretable explanations that align with morpho-constitutional analysis used by clinicians. The model achieves about $90.37\%$ accuracy on a six-class kidney stone dataset and outperforms state-of-the-art methods while providing case-based reasoning explanations that can foster trust and clinical adoption. This work advances clinically usable XAI in urology by linking robust visual features to human-interpretable descriptors and heatmap visualizations.

Abstract

The in-vivo identification of the kidney stone types during an ureteroscopy would be a major medical advance in urology, as it could reduce the time of the tedious renal calculi extraction process, while diminishing infection risks. Furthermore, such an automated procedure would make possible to prescribe anti-recurrence treatments immediately. Nowadays, only few experienced urologists are able to recognize the kidney stone types in the images of the videos displayed on a screen during the endoscopy. Thus, several deep learning (DL) models have recently been proposed to automatically recognize the kidney stone types using ureteroscopic images. However, these DL models are of black box nature whicl limits their applicability in clinical settings. This contribution proposes a case-based reasoning DL model which uses prototypical parts (PPs) and generates local and global descriptors. The PPs encode for each class (i.e., kidney stone type) visual feature information (hue, saturation, intensity and textures) similar to that used by biologists. The PPs are optimally generated due a new loss function used during the model training. Moreover, the local and global descriptors of PPs allow to explain the decisions ("what" information, "where in the images") in an understandable way for biologists and urologists. The proposed DL model has been tested on a database including images of the six most widespread kidney stone types. The overall average classification accuracy was 90.37. When comparing this results with that of the eight other DL models of the kidney stone state-of-the-art, it can be seen that the valuable gain in explanability was not reached at the expense of accuracy which was even slightly increased with respect to that (88.2) of the best method of the literature. These promising and interpretable results also encourage urologists to put their trust in AI-based solutions.

Figures (10)

• Figure 1: Overview of two procedures for determining the type of kidney stones. MCA is an ex-vivo procedure since it requires the extraction of kidney stone fragments from the urinary tract. In Morpho-Constitutional Analysis (MCA), fragments are analyzed by a biologist who determines the type of the kidney stones by a visual inspection followed by a biochemical (FTIR) analysis. On the other hand, automatic MCA (aMCA) uses machine learning-based methods to identify the type of kidney stones using endoscopic images acquired during the ureteroscopy (i.e., in in-vivo). A deep learning (DL) model is trained and exploited to perform real-time inference to assist the urologist in kidney stone recognition. In this contribution, the aMCA is based on explainable Artificial Intelligence (XAI) models that allow to understand the decisions taken by the DL models.
• Figure 2: Illustration of various interpretability levels in the frame of kidney stone type recognition (type "WW" stands for whewellite). (a) Traditional non-interpretable DL models produce a class label without any explanation. (b) Three most common object-level attention maps: i) counterfactual explanation (Guidotti2022) which indicates for each image region the smallest change in feature values that can modify the class label, ii) SHapley Additive exPlanations (SHAP, (Lundberg2017)) which is based on game theory and iii) GradCAM (GradCAM). (c) Decision region delineated with a localization method (Fine_Grained_Zheng_2017_ICCV), (d) Case-based methods quantify the similarity of "decision regions" and prototypical parts (PPs, (ProtoPnet)). (e) PP-descriptors give an indication on the important local and global image features (here hue, saturation, and intensity values in the HSI color space (daul2000) and LBP texture histograms (SERRAT201741).
• Figure 3: Proposed DL model. Input image $x_n$, an acquisition of a weddellite (WD) kidney stone in this example, is processed by CNN $f$ that translates the content of $W \times H$ image patches to a latent feature representation whose discrete volume dimension is given by $\mathbb{R}^{W\times H\times D}$, $W$ and $H$ being respectively the number of the adjacent patches along the columns and lines of $x_n$ (the use of image patches is justified in Section \ref{['Kidney_stone_dataset']}). As sketched in this figure, the tensors $z_{w,h}$ of each image patch correspond to a point (white discs) in an embedding space of 128 dimensions ($D$=128). In this space, the $L2$-distances $d_{z_{w,h},\,p_{m,k}}$ between tensors $z_{w,h}$ and learned PPs (tensors $p_{m,k}$, colored discs) are all assessed. These $L2$-distances are used to compute similarity maps $\mathbf{S}_{m,k}$ (see the "similarity block" of this figure and the map examples given in Fig. \ref{['fig_XAI_methods']}.(d)) which allow to quantify the resemblance of a PP and an image patch. The greatest value of a similarity map acts as similarity score $s_{m,k}$, which indicates to which input image patch the PP is the closest. Finally, the similarity scores $s_{m,k}$ of all PPs are processed by a fully connected (FC) layer to get the logits. A softmax is applied to the logits to determine the class label.
• Figure 4: Similarity visualization: Superimposition of heatmap $\mathbf{H}_{m,k}$ of the most similar PP $p_{m,k}$ on the corresponding region in image $x_n$
• Figure 5: Examples of the six most common kidney stone types: Whewellite (WW), Struvite (STR), Cystine (CYS), Brushite (BRU), Uric Acid (UA), and Weddellite (WD). Surface (SUR) and section (SEC) views of the kidney stone fragments are given in figures (a) and (b), respectively. The complete kidney stone images are given in the upper rows of the figures, while the lower rows represent patches extracted from the images in the upper rows.