Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Seeing Isn't Always Believing: Analysis of Grad-CAM Faithfulness and Localization Reliability in Lung Cancer CT Classification

Teerapong Panboonyuen

TL;DR

This work tackles the trustworthiness of Grad-CAM explanations for lung cancer CT classification across CNNs and Vision Transformers using the IQ-OTH/NCCD dataset. It introduces three quantitative faithfulness metrics—Localization Accuracy, Perturbation-based Faithfulness, and Explanation Consistency—to rigorously assess heatmap reliability across architectures. The results show Grad-CAM is generally faithful for convolutional networks but degrades for Vision Transformers due to non-local attention, with substantial cross-model variability in localization. The study underscores the need for model-aware interpretability and quantitative audits prior to clinical deployment, aiming to move beyond visually appealing heatmaps toward clinically meaningful explanations.

Abstract

Explainable Artificial Intelligence (XAI) techniques, such as Gradient-weighted Class Activation Mapping (Grad-CAM), have become indispensable for visualizing the reasoning process of deep neural networks in medical image analysis. Despite their popularity, the faithfulness and reliability of these heatmap-based explanations remain under scrutiny. This study critically investigates whether Grad-CAM truly represents the internal decision-making of deep models trained for lung cancer image classification. Using the publicly available IQ-OTH/NCCD dataset, we evaluate five representative architectures: ResNet-50, ResNet-101, DenseNet-161, EfficientNet-B0, and ViT-Base-Patch16-224, to explore model-dependent variations in Grad-CAM interpretability. We introduce a quantitative evaluation framework that combines localization accuracy, perturbation-based faithfulness, and explanation consistency to assess Grad-CAM reliability across architectures. Experimental findings reveal that while Grad-CAM effectively highlights salient tumor regions in most convolutional networks, its interpretive fidelity significantly degrades for Vision Transformer models due to non-local attention behavior. Furthermore, cross-model comparisons indicate substantial variability in saliency localization, implying that Grad-CAM explanations may not always correspond to the true diagnostic evidence used by the networks. This work exposes critical limitations of current saliency-based XAI approaches in medical imaging and emphasizes the need for model-aware interpretability methods that are both computationally sound and clinically meaningful. Our findings aim to inspire a more cautious and rigorous adoption of visual explanation tools in medical AI, urging the community to rethink what it truly means to "trust" a model's explanation.

Seeing Isn't Always Believing: Analysis of Grad-CAM Faithfulness and Localization Reliability in Lung Cancer CT Classification

TL;DR

This work tackles the trustworthiness of Grad-CAM explanations for lung cancer CT classification across CNNs and Vision Transformers using the IQ-OTH/NCCD dataset. It introduces three quantitative faithfulness metrics—Localization Accuracy, Perturbation-based Faithfulness, and Explanation Consistency—to rigorously assess heatmap reliability across architectures. The results show Grad-CAM is generally faithful for convolutional networks but degrades for Vision Transformers due to non-local attention, with substantial cross-model variability in localization. The study underscores the need for model-aware interpretability and quantitative audits prior to clinical deployment, aiming to move beyond visually appealing heatmaps toward clinically meaningful explanations.

Abstract

Explainable Artificial Intelligence (XAI) techniques, such as Gradient-weighted Class Activation Mapping (Grad-CAM), have become indispensable for visualizing the reasoning process of deep neural networks in medical image analysis. Despite their popularity, the faithfulness and reliability of these heatmap-based explanations remain under scrutiny. This study critically investigates whether Grad-CAM truly represents the internal decision-making of deep models trained for lung cancer image classification. Using the publicly available IQ-OTH/NCCD dataset, we evaluate five representative architectures: ResNet-50, ResNet-101, DenseNet-161, EfficientNet-B0, and ViT-Base-Patch16-224, to explore model-dependent variations in Grad-CAM interpretability. We introduce a quantitative evaluation framework that combines localization accuracy, perturbation-based faithfulness, and explanation consistency to assess Grad-CAM reliability across architectures. Experimental findings reveal that while Grad-CAM effectively highlights salient tumor regions in most convolutional networks, its interpretive fidelity significantly degrades for Vision Transformer models due to non-local attention behavior. Furthermore, cross-model comparisons indicate substantial variability in saliency localization, implying that Grad-CAM explanations may not always correspond to the true diagnostic evidence used by the networks. This work exposes critical limitations of current saliency-based XAI approaches in medical imaging and emphasizes the need for model-aware interpretability methods that are both computationally sound and clinically meaningful. Our findings aim to inspire a more cautious and rigorous adoption of visual explanation tools in medical AI, urging the community to rethink what it truly means to "trust" a model's explanation.
Paper Structure (32 sections, 5 equations, 4 figures, 1 table)

This paper contains 32 sections, 5 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Approach
  3. Problem Formulation
  4. Grad-CAM for Model Interpretability
  5. Faithfulness Metrics
  6. Localization Accuracy
  7. Perturbation-based Faithfulness
  8. Explanation Consistency
  9. Analysis Across Architectures
  10. Experiments
  11. Dataset
  12. Evaluation Metrics
  13. Training and Validation
  14. Experimental Results
  15. Quantitative Evaluation
  16. ...and 17 more sections

Figures (4)

  • Figure 1: Comparison of Sensitivity, Specificity, and F1-score for different models on the IQ-OTH/NCCD test set.
  • Figure 2: Grad-CAM visualization of five sample test images for models ResNet50, ResNet101, DenseNet161, EfficientNetB0, and ViT-Base-Patch16-224. The first column shows the input image, and subsequent columns show Grad-CAM maps.
  • Figure 3: Additional Grad-CAM visualizations for another set of five test images. The models consistently focus on clinically relevant regions.
  • Figure 4: Confusion matrices of different models on the IQ-OTH/NCCD test set. Rows correspond to true classes, and columns correspond to predicted classes.