DeCoDEx: Confounder Detector Guidance for Improved Diffusion-based Counterfactual Explanations

TL;DR

Deep learning classifiers in medical imaging often rely on confounding artifacts, hindering generalization. DeCoDEx introduces a diffusion-based counterfactual explainer guided by a pre-trained artifact detector, combining $L_{class}$, $L_{det}$, and $L_{perc}$ losses to emphasize causal pathology while suppressing artifacts. On CheXpert with synthetic dots and real support devices, DeCoDEx produces counterfactuals that modify pathology while ignoring artifacts and improves minority subgroup performance when used to augment ERM and Group-DRO classifiers. The approach offers inference-time debiasing with flexible detector integration and does not require retraining debiasing classifiers, advancing reliable explainability in the presence of diverse artifacts.

Abstract

Deep learning classifiers are prone to latching onto dominant confounders present in a dataset rather than on the causal markers associated with the target class, leading to poor generalization and biased predictions. Although explainability via counterfactual image generation has been successful at exposing the problem, bias mitigation strategies that permit accurate explainability in the presence of dominant and diverse artifacts remain unsolved. In this work, we propose the DeCoDEx framework and show how an external, pre-trained binary artifact detector can be leveraged during inference to guide a diffusion-based counterfactual image generator towards accurate explainability. Experiments on the CheXpert dataset, using both synthetic artifacts and real visual artifacts (support devices), show that the proposed method successfully synthesizes the counterfactual images that change the causal pathology markers associated with Pleural Effusion while preserving or ignoring the visual artifacts. Augmentation of ERM and Group-DRO classifiers with the DeCoDEx generated images substantially improves the results across underrepresented groups that are out of distribution for each class. The code is made publicly available at https://github.com/NimaFathi/DeCoDEx.

Figures (5)

• Figure 1: CF explanations for a subject with Pleural Effusion in the presence of an artifact: (a) Chest radiograph of a sick patient: dot artifact, disease pathology; (b) CF image from biased classifier using DDPM (i.e. DeCoDEx without detector) maintains the diseased area but modifies the dot; (c) DeCoDEx CF image modifies the Pleural Effusion area to look healthy as expected huang2022deepwang2017chestx while ignoring the dot artifact.
• Figure 2: DeCoDEx Framework: Generating the counterfactuals (CFs) involves several inference steps. At each step, there are several components: (1) Denoising via unconditional DDPM, (2) pretrained classifier and detector loss, (3) gradient of the classifier, detector and perceptual loss, (4) counterfactual synthesis via sampling and backpropagating loss from black-box classifier and detector. The classifier, detector and unconditional DDPM are all pre-trained components. The resulting CF makes changes to the disease markers while disregarding visual artifacts.
• Figure 3: Majority and Minority subgroup samples from the Dot dataset (top row) and Device dataset (bottom row). Red boxes show the location of the artifacts.
• Figure 4: Qualitative comparison of counterfactual images synthesized via Baseline (i.e. DeCoDEx without detector) and DeCoDEx. For the baseline, most of the changes were made to the spurious correlation but for DeCoDEx visual artifacts were ignored and changes pertained to disease pathology.
• Figure 5: CF explanations for the detector: Removing medical devices from the original images while explaining the detector. Note the disease state is maintained in the counterfactual image.