Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Skull-stripping induces shortcut learning in MRI-based Alzheimer's disease classification

Christian Tinauer, Maximilian Sackl, Rudolf Stollberger, Reinhold Schmidt, Stefan Ropele, Christian Langkammer

TL;DR

This study investigates what drives CNN-based Alzheimer's disease classification from structural MRI by systematically varying preprocessing, notably skull-stripping, and by removing gray–white matter texture through binarization. Using 990 AD and 990 NC images from ADNI across eight input configurations, the authors combine a streamlined 3D CNN with Layer-wise Relevance Propagation, McNemar tests, and spectral relevance analyses to reveal that volume- and contour-based features—enhanced by skull-stripping—are the primary cues, not microstructural texture. The findings demonstrate a robust shortcut learning effect: preprocessing artifacts can become reliable signals, potentially biasing models toward nonpathological cues like brain contours or ventricle sizes. The work emphasizes the necessity of interpretability and rigorous preprocessing validation to avoid clinically misleading conclusions, and suggests incorporating quantitative MRI metrics to improve robustness and generalizability in medical imaging AI systems.

Abstract

Objectives: High classification accuracy of Alzheimer's disease (AD) from structural MRI has been achieved using deep neural networks, yet the specific image features contributing to these decisions remain unclear. In this study, the contributions of T1-weighted (T1w) gray-white matter texture, volumetric information, and preprocessing -- particularly skull-stripping -- were systematically assessed. Methods: A dataset of 990 matched T1w MRIs from AD patients and cognitively normal controls from the ADNI database were used. Preprocessing was varied through skull-stripping and intensity binarization to isolate texture and shape contributions. A 3D convolutional neural network was trained on each configuration, and classification performance was compared using exact McNemar tests with discrete Bonferroni-Holm correction. Feature relevance was analyzed using Layer-wise Relevance Propagation, image similarity metrics, and spectral clustering of relevance maps. Results: Despite substantial differences in image content, classification accuracy, sensitivity, and specificity remained stable across preprocessing conditions. Models trained on binarized images preserved performance, indicating minimal reliance on gray-white matter texture. Instead, volumetric features -- particularly brain contours introduced through skull-stripping -- were consistently used by the models. Conclusions: This behavior reflects a shortcut learning phenomenon, where preprocessing artifacts act as potentially unintended cues. The resulting Clever Hans effect emphasizes the critical importance of interpretability tools to reveal hidden biases and to ensure robust and trustworthy deep learning in medical imaging.

Skull-stripping induces shortcut learning in MRI-based Alzheimer's disease classification

TL;DR

This study investigates what drives CNN-based Alzheimer's disease classification from structural MRI by systematically varying preprocessing, notably skull-stripping, and by removing gray–white matter texture through binarization. Using 990 AD and 990 NC images from ADNI across eight input configurations, the authors combine a streamlined 3D CNN with Layer-wise Relevance Propagation, McNemar tests, and spectral relevance analyses to reveal that volume- and contour-based features—enhanced by skull-stripping—are the primary cues, not microstructural texture. The findings demonstrate a robust shortcut learning effect: preprocessing artifacts can become reliable signals, potentially biasing models toward nonpathological cues like brain contours or ventricle sizes. The work emphasizes the necessity of interpretability and rigorous preprocessing validation to avoid clinically misleading conclusions, and suggests incorporating quantitative MRI metrics to improve robustness and generalizability in medical imaging AI systems.

Abstract

Objectives: High classification accuracy of Alzheimer's disease (AD) from structural MRI has been achieved using deep neural networks, yet the specific image features contributing to these decisions remain unclear. In this study, the contributions of T1-weighted (T1w) gray-white matter texture, volumetric information, and preprocessing -- particularly skull-stripping -- were systematically assessed. Methods: A dataset of 990 matched T1w MRIs from AD patients and cognitively normal controls from the ADNI database were used. Preprocessing was varied through skull-stripping and intensity binarization to isolate texture and shape contributions. A 3D convolutional neural network was trained on each configuration, and classification performance was compared using exact McNemar tests with discrete Bonferroni-Holm correction. Feature relevance was analyzed using Layer-wise Relevance Propagation, image similarity metrics, and spectral clustering of relevance maps. Results: Despite substantial differences in image content, classification accuracy, sensitivity, and specificity remained stable across preprocessing conditions. Models trained on binarized images preserved performance, indicating minimal reliance on gray-white matter texture. Instead, volumetric features -- particularly brain contours introduced through skull-stripping -- were consistently used by the models. Conclusions: This behavior reflects a shortcut learning phenomenon, where preprocessing artifacts act as potentially unintended cues. The resulting Clever Hans effect emphasizes the critical importance of interpretability tools to reveal hidden biases and to ensure robust and trustworthy deep learning in medical imaging.
Paper Structure (22 sections, 16 figures, 7 tables)

This paper contains 22 sections, 16 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Materials and Methods
  3. Imaging Data
  4. Research Groups
  5. Preprocessing
  6. Standard classification network
  7. Training
  8. Heatmapping and relevance-weighted heatmap presentation
  9. Random sampling
  10. Statistical analysis
  11. Heatmap similarity analysis
  12. Results
  13. Model performances
  14. Feature similarities
  15. Discussion
  16. ...and 7 more sections

Figures (16)

  • Figure 1: Input image setups: Left column with (A1) aligned T1w MRI, identical binarized T1w images with the manually selected threshold levels (B1) 13.75%, (C1) 27.50% (C1) and (D1) 41.25%, and in the right column the corresponding skull-stripped versions (A2, B2, C2, D2).
  • Figure 2: Structure of the 3D classifier network. Dimensionalities between layers are the tensor sizes.
  • Figure 3: Mean heatmaps from test images: Left column with (A1) aligned T1w MRI, identical binarized T1w image with threshold levels (B1) 13.75%, (C1) 27.50% (C1) and (D1) 41.25%, and right column with corresponding skull-stripped versions (A2, B2, C2, D2). The mean accuracies of the models are shown in yellow.
  • Figure 4: Standardized mean differences across covariates before and after matching. Only variables age and sex were used for propensity-logit-matching.
  • Figure 5: Residual voxels expressed as a percentage of the brain mask across binarization thresholds. Normalized group differences (NC vs. AD) are shown, with black lines indicating thresholds applied in the model setups.
  • ...and 11 more figures