MiMICRI: Towards Domain-centered Counterfactual Explanations of Cardiovascular Image Classification Models

TL;DR

MiMICRI introduces a domain-centered counterfactual explanation framework for cardiovascular image classification, pairing segmentation-based feature definition with interactive source-target recombination (MorphMix) to produce multiple domain-relevant explanations. Implemented as a Python visualization package and demonstrated on UK Biobank cardiac MRIs with a hypertension predictor, the approach yields explanations that align with medical knowledge but raises concerns about clinical plausibility and generalizability. Expert evaluations show improved interpretability and nuanced reasoning via segment-level replacements, while highlighting interdependencies among cardiac structures that can limit plausibility. The work emphasizes trustworthiness and domain augmentation, calling for careful deployment and further research to generalize domain-centered XAI methods across healthcare contexts.

Abstract

The recent prevalence of publicly accessible, large medical imaging datasets has led to a proliferation of artificial intelligence (AI) models for cardiovascular image classification and analysis. At the same time, the potentially significant impacts of these models have motivated the development of a range of explainable AI (XAI) methods that aim to explain model predictions given certain image inputs. However, many of these methods are not developed or evaluated with domain experts, and explanations are not contextualized in terms of medical expertise or domain knowledge. In this paper, we propose a novel framework and python library, MiMICRI, that provides domain-centered counterfactual explanations of cardiovascular image classification models. MiMICRI helps users interactively select and replace segments of medical images that correspond to morphological structures. From the counterfactuals generated, users can then assess the influence of each segment on model predictions, and validate the model against known medical facts. We evaluate this library with two medical experts. Our evaluation demonstrates that a domain-centered XAI approach can enhance the interpretability of model explanations, and help experts reason about models in terms of relevant domain knowledge. However, concerns were also surfaced about the clinical plausibility of the counterfactuals generated. We conclude with a discussion on the generalizability and trustworthiness of the MiMICRI framework, as well as the implications of our findings on the development of domain-centered XAI methods for model interpretability in healthcare contexts.

Figures (6)

• Figure 1: A high-level summary of the MiMICRI framework. To explain a classification MODEL, users can identify domain-relevant semantic image segments in each image in the data set, then replace segments in a target image with corresponding segments from a source image. This creates a recombined image. If the MODEL predicts that the recombined image has an alternate label to the target image, this recombined image is a counterfactual, and we can conclude that the replaced segment changed the MODEL prediction.
• Figure 2: The detailed MiMICRI framework and corresponding visualization modules. Top: Users interactively select source and target images or videos. They can also select combinations of segmented features to be replaced. Selected segmented areas (e.g. circular shapes in orange for LV Cavity) are overlaid on top of MRIs at their corresponding positions. Bottom: Selected morphological segments from target(s) are masked and replaced with corresponding segments from source(s). We implemented the MorphMix method to do this. New predicted labels are generated for the recombined images or videos. Users interactively inspect the model by viewing the counterfactuals generated for each replaced segment.
• Figure 3: The MiMICRI selector module. In this module, users can 1) view a target image or video, 2) select segments to replace, 3) select source images by demographic by adding filter variables, 4) view and modify the range of selected values for each demographic filter, 5) view the selected subset in an icicle plot and corresponding unit visualization, and 6) view detailed source images or videos by dragging the brush over the unit visualization. In both the top and bottom panel, the visibility of the overlay can be toggled. If the files are videos, the videos can be paused. The panels can be resized.
• Figure 4: Left: A single frame from a source cardiac MRI. Middle: A single frame from a target cardiac MRI with all 3 cardiac segments to be masked. Right: A recombined frame where pixels from the 3 corresponding cardiac segments in the source image were copied into the target image. Note how, with the exception of the replaced segments, the recombined image is identical to the target.
• Figure 5: Expert feedback for four recombined images. Original source and target images were omitted for a compact layout. Left: Two recombined MRIs and corresponding segmentation that were acceptable to experts. Middle: A recombined image with a double wall in the RV. "Though that may not affect segmentation, it would likely affect any whole-image analysis" (E1). Right: A particularly egregious example where physiological features were disordered and segmented regions contain artifacts.