Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Caption-Driven Explainability: Probing CNNs for Bias via CLIP

Patrick Koller, Amil V. Dravid, Guido M. Schuster, Aggelos K. Katsaggelos

TL;DR

The paper tackles robustness and bias in CNNs by introducing a caption-based XAI method that leverages CLIP and a novel network-surgery approach to reveal the dominant concepts driving predictions. It transfers a standalone model into CLIP via activation-map swapping, guided by activation-matching and correlation, to produce caption-derived explanations that reflect high-level concepts. Experiments on colored MNIST show the method can identify color biases, enable debiasing through grayscale pre-processing, and shift the dominant concept toward shape, illustrating improved robustness compared to saliency maps. This approach provides a principled way to understand and rectify covariate shifts before deploying vision models, with potential impact in safety-critical domains like medicine.

Abstract

Robustness has become one of the most critical problems in machine learning (ML). The science of interpreting ML models to understand their behavior and improve their robustness is referred to as explainable artificial intelligence (XAI). One of the state-of-the-art XAI methods for computer vision problems is to generate saliency maps. A saliency map highlights the pixel space of an image that excites the ML model the most. However, this property could be misleading if spurious and salient features are present in overlapping pixel spaces. In this paper, we propose a caption-based XAI method, which integrates a standalone model to be explained into the contrastive language-image pre-training (CLIP) model using a novel network surgery approach. The resulting caption-based XAI model identifies the dominant concept that contributes the most to the models prediction. This explanation minimizes the risk of the standalone model falling for a covariate shift and contributes significantly towards developing robust ML models. Our code is available at https://github.com/patch0816/caption-driven-xai

Caption-Driven Explainability: Probing CNNs for Bias via CLIP

TL;DR

The paper tackles robustness and bias in CNNs by introducing a caption-based XAI method that leverages CLIP and a novel network-surgery approach to reveal the dominant concepts driving predictions. It transfers a standalone model into CLIP via activation-map swapping, guided by activation-matching and correlation, to produce caption-derived explanations that reflect high-level concepts. Experiments on colored MNIST show the method can identify color biases, enable debiasing through grayscale pre-processing, and shift the dominant concept toward shape, illustrating improved robustness compared to saliency maps. This approach provides a principled way to understand and rectify covariate shifts before deploying vision models, with potential impact in safety-critical domains like medicine.

Abstract

Robustness has become one of the most critical problems in machine learning (ML). The science of interpreting ML models to understand their behavior and improve their robustness is referred to as explainable artificial intelligence (XAI). One of the state-of-the-art XAI methods for computer vision problems is to generate saliency maps. A saliency map highlights the pixel space of an image that excites the ML model the most. However, this property could be misleading if spurious and salient features are present in overlapping pixel spaces. In this paper, we propose a caption-based XAI method, which integrates a standalone model to be explained into the contrastive language-image pre-training (CLIP) model using a novel network surgery approach. The resulting caption-based XAI model identifies the dominant concept that contributes the most to the models prediction. This explanation minimizes the risk of the standalone model falling for a covariate shift and contributes significantly towards developing robust ML models. Our code is available at https://github.com/patch0816/caption-driven-xai
Paper Structure (8 sections, 4 equations, 5 figures)

This paper contains 8 sections, 4 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Proposed method
  3. Architecture
  4. Statistics
  5. Activation matching
  6. Network surgery
  7. Experiments
  8. Conclusion

Figures (5)

  • Figure 1: The standalone ResNet-50 model (Red) consists of an image encoder and a fully-connected linear classifier. The learning curves indicate a low bias, low variance ML model. Whether the ML model is biased without using XAI cannot be stated with certainty.
  • Figure 2: CLIP is the core component of the proposed caption-based XAI model. Using CLIP's text encoder (Purple) and image encoder (Green), the resulting embedding similarities reveal what CLIP's image encoder is focusing on by using the captions. The largest embedding similarities are highlighted (Blue). The network surgery process allows integration of any standalone model into CLIP, so CLIP can explain what the standalone image encoder (Red) focuses on.
  • Figure 3: All activation maps within the four convolutional layers in the CLIP image encoder and 49 convolutional layers in the standalone model are available for swapping.
  • Figure 4: Color is the dominant concept for the standalone model and shape is the dominant concept for the unbiased standalone model.
  • Figure 5: The caption-based XAI method identifies the color feature as the dominant feature. Removing the color feature and retraining makes the standalone model robust. The captions of the caption-based XAI model identify the shift from the color to the shape feature.