Here Comes the Explanation: A Shapley Perspective on Multi-contrast Medical Image Segmentation

TL;DR

The results demonstrate that Shapley analysis provides valuable insights into different models' behavior used for tumor segmentation, and a bias for U-Net towards over-weighing T1-contrast and FLAIR, while Swin-UNETR provided a cross-contrast understanding with balanced Shapley distribution.

Abstract

Deep learning has been successfully applied to medical image segmentation, enabling accurate identification of regions of interest such as organs and lesions. This approach works effectively across diverse datasets, including those with single-image contrast, multi-contrast, and multimodal imaging data. To improve human understanding of these black-box models, there is a growing need for Explainable AI (XAI) techniques for model transparency and accountability. Previous research has primarily focused on post hoc pixel-level explanations, using methods gradient-based and perturbation-based apporaches. These methods rely on gradients or perturbations to explain model predictions. However, these pixel-level explanations often struggle with the complexity inherent in multi-contrast magnetic resonance imaging (MRI) segmentation tasks, and the sparsely distributed explanations have limited clinical relevance. In this study, we propose using contrast-level Shapley values to explain state-of-the-art models trained on standard metrics used in brain tumor segmentation. Our results demonstrate that Shapley analysis provides valuable insights into different models' behavior used for tumor segmentation. We demonstrated a bias for U-Net towards over-weighing T1-contrast and FLAIR, while Swin-UNETR provided a cross-contrast understanding with balanced Shapley distribution.

Figures (4)

• Figure 1: (a) An example of tumor segmentation from multi-contrast MRI. The decision process is not always intuitive because the model does not explain which contrast contributes to the decision, as redundant information can be observed between image contrasts. (b) Our proposed Contrast-level shapley value aims to provide a cross-contrast level explanation which provides a global understanding of the multi-contrast image segmentation.
• Figure 2: Clustering results on (a) $\cup_{f=1}^{5} \cup_{j=1}^{J_f} \{ \mathbf{S}_j^{\text{Unet}, f}(\text{Dice}) \}$and (b) $\cup_{f=1}^{5} \cup_{j=1}^{J_f} \{ \mathbf{S}_j^{\text{Swin-UNETR}, f}(\text{Dice}) \}$ are visualized using UMAP for dimensionality reduction. The color represents the Dice score; the size of the dot is used to differentiate between cluster labels.
• Figure 3: The contrast-level Shapley values for all folds are computed based on the Dice score in each model. Panels (a) and (b) display the case of UNet and Swin-UNETR models, respectively.
• Figure 4: Case comparison where Swin-UNETR outperforms U-Net. For the first four columns, from top to bottom, display: Ground truth, U-Net predictions, and Swin-UNETR predictions. For the last four columns, from top to bottom, display: input images, model explanations for U-Net (explanation (a) and (b)), and Swin-UNETR predictions (explanation (a) and (b)), where (a) shows GradCAM explanation for each contrast and (b) presents the proposed constrast-level Shapley values.