Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

This changes to that : Combining causal and non-causal explanations to generate disease progression in capsule endoscopy

Anuja Vats, Ahmed Mohammed, Marius Pedersen, Nirmalie Wiratunga

TL;DR

The paper tackles the need for transparent reasoning in high-stakes medical imaging by proposing a unified explanation framework that combines causal (counterfactual and semifactual) and non-causal (saliency-based) explanations. It leverages StyleGAN2 latent space for attribute-driven exploration and SeFA-based unsupervised attribute discovery to construct an explanation set X = {i_sm, i_cf, i_sf} along a chosen UC biomarker path, with saliency derived from semantic, attribute-guided directional derivatives. Counterfactual and semifactual examples are selected based on classifier outputs near the decision boundary, enabling visual, interpretable progressions alongside the saliency map. Evaluations on Wireless Capsule Endoscopy data using the Softmax Information Curve show competitive results, demonstrating the approach's potential to support UC prognosis and broader, high-stakes medical decision making with interpretable, causally meaningful explanations.

Abstract

Due to the unequivocal need for understanding the decision processes of deep learning networks, both modal-dependent and model-agnostic techniques have become very popular. Although both of these ideas provide transparency for automated decision making, most methodologies focus on either using the modal-gradients (model-dependent) or ignoring the model internal states and reasoning with a model's behavior/outcome (model-agnostic) to instances. In this work, we propose a unified explanation approach that given an instance combines both model-dependent and agnostic explanations to produce an explanation set. The generated explanations are not only consistent in the neighborhood of a sample but can highlight causal relationships between image content and the outcome. We use Wireless Capsule Endoscopy (WCE) domain to illustrate the effectiveness of our explanations. The saliency maps generated by our approach are comparable or better on the softmax information score.

This changes to that : Combining causal and non-causal explanations to generate disease progression in capsule endoscopy

TL;DR

The paper tackles the need for transparent reasoning in high-stakes medical imaging by proposing a unified explanation framework that combines causal (counterfactual and semifactual) and non-causal (saliency-based) explanations. It leverages StyleGAN2 latent space for attribute-driven exploration and SeFA-based unsupervised attribute discovery to construct an explanation set X = {i_sm, i_cf, i_sf} along a chosen UC biomarker path, with saliency derived from semantic, attribute-guided directional derivatives. Counterfactual and semifactual examples are selected based on classifier outputs near the decision boundary, enabling visual, interpretable progressions alongside the saliency map. Evaluations on Wireless Capsule Endoscopy data using the Softmax Information Curve show competitive results, demonstrating the approach's potential to support UC prognosis and broader, high-stakes medical decision making with interpretable, causally meaningful explanations.

Abstract

Due to the unequivocal need for understanding the decision processes of deep learning networks, both modal-dependent and model-agnostic techniques have become very popular. Although both of these ideas provide transparency for automated decision making, most methodologies focus on either using the modal-gradients (model-dependent) or ignoring the model internal states and reasoning with a model's behavior/outcome (model-agnostic) to instances. In this work, we propose a unified explanation approach that given an instance combines both model-dependent and agnostic explanations to produce an explanation set. The generated explanations are not only consistent in the neighborhood of a sample but can highlight causal relationships between image content and the outcome. We use Wireless Capsule Endoscopy (WCE) domain to illustrate the effectiveness of our explanations. The saliency maps generated by our approach are comparable or better on the softmax information score.
Paper Structure (4 sections, 1 equation, 6 figures, 1 algorithm)

This paper contains 4 sections, 1 equation, 6 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methodology
  3. Results
  4. Conclusion

Figures (6)

  • Figure 1: Figure shows $i_a$ and the corresponding directional derivatives. The derivatives expose the semantic similarity between the query and it's neighbors. We use this similarity to weigh in the contribution of each neighbor towards the saliency map.
  • Figure 2: The approach explains a query image along the ulcer attribute path together with a semifactual and counterfactual along the same path. Here the query exhibits an abnormality with inflammation. Even with inflammation reduced down to as in (c) the prediction would still be abnormal (semifactual). However, if only the visual signs change from (c) to as in (b), the prediction would be normal (counterfactual).
  • Figure 3: Images in $i_a$ along attribute $a$. Top left corner shows softmax score. Notice how apart from effected region (for attribute a), other regions in the image undergo only minimal changes. As a result, the generated explanations are consistent in the locality of a query.
  • Figure 4: Qualitative comparison of saliency maps between our approach and other approaches. Integrated Gradients (IG) sundararajan2017IG, Guided integrated gradients kapishnikov2021GIG, SmoothGrad smilkov2017smoothgrad
  • Figure 5: Figure shows $\mathcal{X}$ generated with this approach on different query images (column 3). Best viewed in color.
  • ...and 1 more figures