Quantitative Evaluation of the Saliency Map for Alzheimer's Disease Classifier with Anatomical Segmentation

TL;DR

This work tackles the challenge of evaluating saliency maps for Alzheimer's disease classifiers amid heterogeneity by introducing a brain-wide evaluation metric, the Brain Volume Change Score ($VCS$), computed via Pearson correlations between region-wise saliency and longitudinal volume changes using anatomical segmentation. It allocates saliency to 95 brain regions and demonstrates that a higher $VCS$ corresponds to saliency maps more aligned with AD pathology, beyond hippocampal focus. The authors further show that gradient-based adversarial training, specifically FGSM and stochastic masking, along with a consistency loss, improves $VCS$ and, in several cross-dataset tests, enhances classification metrics. The proposed framework provides a practical, region-level interpretability tool for AD classifiers and is easily extendable to other medical-imaging tasks with follow-up data.

Abstract

Saliency maps have been widely used to interpret deep learning classifiers for Alzheimer's disease (AD). However, since AD is heterogeneous and has multiple subtypes, the pathological mechanism of AD remains not fully understood and may vary from patient to patient. Due to the lack of such understanding, it is difficult to comprehensively and effectively assess the saliency map of AD classifier. In this paper, we utilize the anatomical segmentation to allocate saliency values into different brain regions. By plotting the distributions of saliency maps corresponding to AD and NC (Normal Control), we can gain a comprehensive view of the model's decisions process. In order to leverage the fact that the brain volume shrinkage happens in AD patients during disease progression, we define a new evaluation metric, brain volume change score (VCS), by computing the average Pearson correlation of the brain volume changes and the saliency values of a model in different brain regions for each patient. Thus, the VCS metric can help us gain some knowledge of how saliency maps resulting from different models relate to the changes of the volumes across different regions in the whole brain. We trained candidate models on the ADNI dataset and tested on three different datasets. Our results indicate: (i) models with higher VCSs tend to demonstrate saliency maps with more details relevant to the AD pathology, (ii) using gradient-based adversarial training strategies such as FGSM and stochastic masking can improve the VCSs of the models.

Figures (2)

• Figure 1: A. Fast Gradient Sign Method(FGSM). After adding perturbation several times, NC patient is then misclassified by the target network as AD when it is still clearly NC. B.Stochastic Masking. For each image, model has a probability of $\tau$ to be masked by TopK blocks with highest gradients, while 1-$\tau$ to be masked with random blocks. C.Architecture of the training framework. D. T1-weighted MR image, Saliency map, and Segmentation map in x-y-z axis. E. The averaged distribution of model's gradient across brain regions for all AD patients, computed by overlapping of Saliency map and segmentation map. F. The averaged distribution of model's gradient across brain regions for all non-AD patients.
• Figure 2: A. Saliency map of Baseline 3d-CNN. B. Saliency map of 3d-CNN+FGSM+mask. C. Scatter plot of $\Delta V_{n,i}$ vs $S_{n, i}$ of baseline model. D. Scatter plot of $\Delta V_{n,i}$ vs $S_{n, i}$ of FGSM+mask model. We can see a higher correlation (red line denotes correlation equals 1) especially when $\Delta V_{n,i}$ is high (shown in black box). E. Box plot of all $\mathcal{P}_{i}$ for three candidate models.