Developing Explainable Machine Learning Model using Augmented Concept Activation Vector

TL;DR

The paper addresses explainability in medical image analysis by linking high-level radiomic concepts to model decisions. It introduces Augmented Concept Activation Vector (ACAV), which augments inputs with concept patterns and measures their impact on neural activations using a single network, offering an alternative to TCAV that preserves context. The approach quantifies concept influence via activation deviation $\Delta V$ and an imbalance entropy $H$, and demonstrates effectiveness on fundus imaging for retinopathy and brain MRI tumor-size concepts, highlighting the ability to isolate the contribution of rarer patterns. This method has practical significance for validating clinically relevant features and integrating radiomic knowledge into classifiers, with potential applicability beyond the tested datasets.

Abstract

Machine learning models use high dimensional feature spaces to map their inputs to the corresponding class labels. However, these features often do not have a one-to-one correspondence with physical concepts understandable by humans, which hinders the ability to provide a meaningful explanation for the decisions made by these models. We propose a method for measuring the correlation between high-level concepts and the decisions made by a machine learning model. Our method can isolate the impact of a given high-level concept and accurately measure it quantitatively. Additionally, this study aims to determine the prevalence of frequent patterns in machine learning models, which often occur in imbalanced datasets. We have successfully applied the proposed method to fundus images and managed to quantitatively measure the impact of radiomic patterns on the model decisions.

Figures (8)

• Figure 1: Schematic diagram of the proposed method.
• Figure 2: Healthy images augmented with cotton-wool radiomic pattern.
• Figure 3: Healthy images augmented with cotton-wool and fatty dots radiomic patterns.
• Figure 4: Images with bleeding pattern augmented with fatty dots radiomic pattern.
• Figure 5: Schematic diagram of the classifying model.