Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Instance-level quantitative saliency in multiple sclerosis lesion segmentation

Federico Spagnolo, Nataliia Molchanova, Meritxell Bach Cuadra, Mario Ocampo Pineda, Lester Melie-Garcia, Cristina Granziera, Vincent Andrearczyk, Adrien Depeursinge

TL;DR

Two architecture-agnostic XAI methods are introduced that provide quantitative instance-level explanations for semantic segmentation and support clinically meaningful interpretation of model decisions.

Abstract

Explainable artificial intelligence (XAI) methods have been proposed to interpret model decisions in classification and, more recently, in semantic segmentation. However, instance-level XAI for semantic segmentation, namely explanations focused on a single object among multiple instances of the same class, remains largely unexplored. Such explanations are particularly important in multi-lesional diseases to understand what drives the detection and contouring of a specific lesion. We propose instance-level explanation maps for semantic segmentation by extending SmoothGrad and Grad-CAM++ to obtain quantitative instance saliency. These methods were applied to the segmentation of white matter lesions (WMLs), a magnetic resonance imaging biomarker in multiple sclerosis. We used 4023 FLAIR and MPRAGE MRI scans from 687 patients collected at the University Hospital of Basel, Switzerland, with WML masks annotated by four expert clinicians. Three deep learning architectures, a 3D U-Net, nnU-Net, and Swin UNETR, were trained and evaluated, achieving normalized Dice scores of 0.71, 0.78, and 0.80, respectively. Instance saliency maps showed that the models relied primarily on FLAIR rather than MPRAGE for WML segmentation, with positive saliency inside lesions and negative saliency in their immediate neighborhood, consistent with clinical practice. Peak saliency values differed significantly across correct and incorrect predictions, suggesting that quantitative instance saliency may help identify segmentation errors. In conclusion, we introduce two architecture-agnostic XAI methods that provide quantitative instance-level explanations for semantic segmentation and support clinically meaningful interpretation of model decisions.

Instance-level quantitative saliency in multiple sclerosis lesion segmentation

TL;DR

Two architecture-agnostic XAI methods are introduced that provide quantitative instance-level explanations for semantic segmentation and support clinically meaningful interpretation of model decisions.

Abstract

Explainable artificial intelligence (XAI) methods have been proposed to interpret model decisions in classification and, more recently, in semantic segmentation. However, instance-level XAI for semantic segmentation, namely explanations focused on a single object among multiple instances of the same class, remains largely unexplored. Such explanations are particularly important in multi-lesional diseases to understand what drives the detection and contouring of a specific lesion. We propose instance-level explanation maps for semantic segmentation by extending SmoothGrad and Grad-CAM++ to obtain quantitative instance saliency. These methods were applied to the segmentation of white matter lesions (WMLs), a magnetic resonance imaging biomarker in multiple sclerosis. We used 4023 FLAIR and MPRAGE MRI scans from 687 patients collected at the University Hospital of Basel, Switzerland, with WML masks annotated by four expert clinicians. Three deep learning architectures, a 3D U-Net, nnU-Net, and Swin UNETR, were trained and evaluated, achieving normalized Dice scores of 0.71, 0.78, and 0.80, respectively. Instance saliency maps showed that the models relied primarily on FLAIR rather than MPRAGE for WML segmentation, with positive saliency inside lesions and negative saliency in their immediate neighborhood, consistent with clinical practice. Peak saliency values differed significantly across correct and incorrect predictions, suggesting that quantitative instance saliency may help identify segmentation errors. In conclusion, we introduce two architecture-agnostic XAI methods that provide quantitative instance-level explanations for semantic segmentation and support clinically meaningful interpretation of model decisions.
Paper Structure (23 sections, 10 equations, 15 figures)

This paper contains 23 sections, 10 equations, 15 figures.

Table of Contents

  1. Introduction
  2. Methods
  3. Results
  4. Discussion
  5. Conclusion
  6. Data Availability
  7. Acknowledgements
  8. Author contributions statement
  9. Additional information

Figures (15)

  • Figure 1: (a) FLAIR MRI presenting WM lesions segmented as separate entities, and (b) example of instance segmentation inspired by Varatharasan et al.varatharasan.
  • Figure 2: Input-output dimensions for a classification (top) and a semantic segmentation (bottom) task.
  • Figure 3: (a) The output of a semantic segmentation network showing several instances of the considered class. SmoothGrad (b) and Grad-CAM (c) applied to all the spatial predictions. (d) How can we explain the segmentation of a particular lesion of interest (e.g., the yellow instance)?
  • Figure 4: Overview of the proposed adaptation of SG to segmentation.
  • Figure 5: Overview of Grad-CAM++, generating a class-level explanation heatmap, similarly to Vinogradova et al. vinogradova2020, adapted for segmentation.
  • ...and 10 more figures