Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TACE: Tumor-Aware Counterfactual Explanations

Eleonora Beatrice Rossi, Eleonora Lopez, Danilo Comminiello

TL;DR

This study proposes Tumor Aware Counterfactual Explanations (TACE), a framework designed to generate reliable counterfactual explanations for medical images that far exceeds existing state-of-the-art techniques in quality, faithfulness, and generation speed of counterfactuals.

Abstract

The application of deep learning in medical imaging has significantly advanced diagnostic capabilities, enhancing both accuracy and efficiency. Despite these benefits, the lack of transparency in these AI models, often termed "black boxes," raises concerns about their reliability in clinical settings. Explainable AI (XAI) aims to mitigate these concerns by developing methods that make AI decisions understandable and trustworthy. In this study, we propose Tumor Aware Counterfactual Explanations (TACE), a framework designed to generate reliable counterfactual explanations for medical images. Unlike existing methods, TACE focuses on modifying tumor-specific features without altering the overall organ structure, ensuring the faithfulness of the counterfactuals. We achieve this by including an additional step in the generation process which allows to modify only the region of interest (ROI), thus yielding more reliable counterfactuals as the rest of the organ remains unchanged. We evaluate our method on mammography images and brain MRI. We find that our method far exceeds existing state-of-the-art techniques in quality, faithfulness, and generation speed of counterfactuals. Indeed, more faithful explanations lead to a significant improvement in classification success rates, with a 10.69% increase for breast cancer and a 98.02% increase for brain tumors. The code of our work is available at https://github.com/ispamm/TACE.

TACE: Tumor-Aware Counterfactual Explanations

TL;DR

This study proposes Tumor Aware Counterfactual Explanations (TACE), a framework designed to generate reliable counterfactual explanations for medical images that far exceeds existing state-of-the-art techniques in quality, faithfulness, and generation speed of counterfactuals.

Abstract

The application of deep learning in medical imaging has significantly advanced diagnostic capabilities, enhancing both accuracy and efficiency. Despite these benefits, the lack of transparency in these AI models, often termed "black boxes," raises concerns about their reliability in clinical settings. Explainable AI (XAI) aims to mitigate these concerns by developing methods that make AI decisions understandable and trustworthy. In this study, we propose Tumor Aware Counterfactual Explanations (TACE), a framework designed to generate reliable counterfactual explanations for medical images. Unlike existing methods, TACE focuses on modifying tumor-specific features without altering the overall organ structure, ensuring the faithfulness of the counterfactuals. We achieve this by including an additional step in the generation process which allows to modify only the region of interest (ROI), thus yielding more reliable counterfactuals as the rest of the organ remains unchanged. We evaluate our method on mammography images and brain MRI. We find that our method far exceeds existing state-of-the-art techniques in quality, faithfulness, and generation speed of counterfactuals. Indeed, more faithful explanations lead to a significant improvement in classification success rates, with a 10.69% increase for breast cancer and a 98.02% increase for brain tumors. The code of our work is available at https://github.com/ispamm/TACE.
Paper Structure (11 sections, 4 equations, 3 figures, 3 tables)

This paper contains 11 sections, 4 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Proposed method
  4. Experimental evaluation
  5. Datasets
  6. Experimental setup
  7. Metrics
  8. Experimental results
  9. Quantitative analysis
  10. Qualitative analysis
  11. Conclusions

Figures (3)

  • Figure 1: The TACE framework involves three main steps. First, the query image is processed by the encoder (E), which converts it into a set of initial blobs representing various spatial and style parameters. Next, the generative network (G) refines these initial blobs to produce a reconstruction of the query image. Finally, the refined blobs are modified to generate counterfactual explanations. The blobs with the most significant changes are identified and adjusted to maintain high similarity to the original image, providing clear and meaningful counterfactual explanations.
  • Figure 2: Counterfactuals of brain MRI generated with StylEx and with TACE. It is clear that StylEx modifies the colors, the internal structure of the brain/breast, and its orientation. In contrast, TACE generates faithful counterfactuals.
  • Figure 3: Comparison of OCTET and TACE. In brain MRIs, OCTET modifies the internal brain structure, adding details not present in the original image. In contrast, TACE limits modifications to the tumor area, making the changes more imperceptible and targeted. In mammographies, the tumor area added by OCTET covers nearly the entire breast, whereas TACE modifications are much more circumscribed and precise.