Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tree-Based Leakage Inspection and Control in Concept Bottleneck Models

Angelos Ragkousis, Sonali Parbhoo

TL;DR

This paper introduces a novel approach for training both joint and sequential CBMs that allows us to identify and control leakage using decision trees, and shows that soft leaky CBMs extend the decision paths of hard CBMs, particularly in cases where concept information is incomplete.

Abstract

As AI models grow larger, the demand for accountability and interpretability has become increasingly critical for understanding their decision-making processes. Concept Bottleneck Models (CBMs) have gained attention for enhancing interpretability by mapping inputs to intermediate concepts before making final predictions. However, CBMs often suffer from information leakage, where additional input data, not captured by the concepts, is used to improve task performance, complicating the interpretation of downstream predictions. In this paper, we introduce a novel approach for training both joint and sequential CBMs that allows us to identify and control leakage using decision trees. Our method quantifies leakage by comparing the decision paths of hard CBMs with their soft, leaky counterparts. Specifically, we show that soft leaky CBMs extend the decision paths of hard CBMs, particularly in cases where concept information is incomplete. Using this insight, we develop a technique to better inspect and manage leakage, isolating the subsets of data most affected by this. Through synthetic and real-world experiments, we demonstrate that controlling leakage in this way not only improves task accuracy but also yields more informative and transparent explanations.

Tree-Based Leakage Inspection and Control in Concept Bottleneck Models

TL;DR

This paper introduces a novel approach for training both joint and sequential CBMs that allows us to identify and control leakage using decision trees, and shows that soft leaky CBMs extend the decision paths of hard CBMs, particularly in cases where concept information is incomplete.

Abstract

As AI models grow larger, the demand for accountability and interpretability has become increasingly critical for understanding their decision-making processes. Concept Bottleneck Models (CBMs) have gained attention for enhancing interpretability by mapping inputs to intermediate concepts before making final predictions. However, CBMs often suffer from information leakage, where additional input data, not captured by the concepts, is used to improve task performance, complicating the interpretation of downstream predictions. In this paper, we introduce a novel approach for training both joint and sequential CBMs that allows us to identify and control leakage using decision trees. Our method quantifies leakage by comparing the decision paths of hard CBMs with their soft, leaky counterparts. Specifically, we show that soft leaky CBMs extend the decision paths of hard CBMs, particularly in cases where concept information is incomplete. Using this insight, we develop a technique to better inspect and manage leakage, isolating the subsets of data most affected by this. Through synthetic and real-world experiments, we demonstrate that controlling leakage in this way not only improves task accuracy but also yields more informative and transparent explanations.
Paper Structure (47 sections, 8 equations, 18 figures, 5 tables, 2 algorithms)

This paper contains 47 sections, 8 equations, 18 figures, 5 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Concept Bottleneck Models and Information Leakage.
  4. Explainable Label Predictors.
  5. Preliminaries
  6. Problem Setting.
  7. Concept Bottleneck Models.
  8. Leakage in CBMs.
  9. Tree-Based Leakage Inspection and Control
  10. The Mixed Sequential CBM (MCBM-Seq) Training Algorithm.
  11. Identifying samples corresponding to certain concepts.
  12. Calibrating the concept probabilities to prevent overconfident predictions.
  13. Training a mixed sequential CBM for each decision path.
  14. Merging the sub-trees to the global tree.
  15. Quantifying Information Leakage with Trees
  16. ...and 32 more sections

Figures (18)

  • Figure 1: An overview of the Mixed CBM Algorithm (MCBM). Step 1: Two independent networks, a concept predictor with calibrated probability outputs and a decision tree label predictor (global tree) are trained. Step 2: A Sequential CBM (sub-tree) with mixed concept representations (Mixed CBM) further splits the leaf nodes of the global tree that present missing concept information and are prone to Leakage. Step 3: All trees are merged for global Leakage Inspection.
  • Figure 2: Summary of the Decision Making Process of The Mixed Sequential CBM Algorithm (MCBM-Seq) when classifying an image with annotated concepts. The process is described intuitively as a conversation between the Tree label predictor and two entities that provide the input concepts: the Human Annotator and the Concept Predictor. The concept probabilities are used to specialise the decision process only when the available annotated concepts are not sufficient.
  • Figure 2: Task and Concept Accuracy across different datasets and CBM training methods. Our Mixed CBM methods are comparable with current approaches in overall performance.
  • Figure 3: The MCBM-Seq algorithm for a reduced Morpho-MNIST dataset with digits 6, 8 and 9 and concepts "length", "thickness" and "width". The final tree merged the sub-trees is shown. If a sub-tree is found, it replaces the leaf node of the hard CBM and is highlighted in a red box. The remaining leaf nodes are unaffected by leakage. This architecture allows us to both inspect and restrict leakage only to subsets with missing concept information.
  • Figure 4: Leakage with Concept Completeness (CUB).
  • ...and 13 more figures

Theorems & Definitions (1)

  • Definition 3.1