Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantifying the Accuracy-Interpretability Trade-Off in Concept-Based Sidechannel Models

David Debot, Giuseppe Marra

TL;DR

The paper tackles the problem of balancing predictive accuracy with interpretability in Concept Sidechannel Models (CSMs), which extend Concept Bottleneck Models (CBMs) by using a sidechannel to boost performance. It introduces a unified probabilistic meta-model and the Sidechannel Independence Score (SIS) to quantify how much a model relies on the sidechannel, along with SIS regularization to explicitly minimize this reliance. The authors analyze expressivity in bottleneck versus default modes and demonstrate, across CelebA and MNIST-Addition, that SIS regularization yields substantial gains in representation interpretability and intervenability while preserving accuracy, with models like CMR and its enhanced variant achieving strong performance. This framework provides a principled toolkit for constructing CSMs that navigate the accuracy–interpretability trade-off and offers practical guidance for training and evaluation. The work advances both theory and practice by enabling explicit control over how much uninterpretable side information influences predictions, with broad implications for interpretable AI deployment.

Abstract

Concept Bottleneck Models (CBNMs) are deep learning models that provide interpretability by enforcing a bottleneck layer where predictions are based exclusively on human-understandable concepts. However, this constraint also restricts information flow and often results in reduced predictive accuracy. Concept Sidechannel Models (CSMs) address this limitation by introducing a sidechannel that bypasses the bottleneck and carry additional task-relevant information. While this improves accuracy, it simultaneously compromises interpretability, as predictions may rely on uninterpretable representations transmitted through sidechannels. Currently, there exists no principled technique to control this fundamental trade-off. In this paper, we close this gap. First, we present a unified probabilistic concept sidechannel meta-model that subsumes existing CSMs as special cases. Building on this framework, we introduce the Sidechannel Independence Score (SIS), a metric that quantifies a CSM's reliance on its sidechannel by contrasting predictions made with and without sidechannel information. We propose SIS regularization, which explicitly penalizes sidechannel reliance to improve interpretability. Finally, we analyze how the expressivity of the predictor and the reliance of the sidechannel jointly shape interpretability, revealing inherent trade-offs across different CSM architectures. Empirical results show that state-of-the-art CSMs, when trained solely for accuracy, exhibit low representation interpretability, and that SIS regularization substantially improves their interpretability, intervenability, and the quality of learned interpretable task predictors. Our work provides both theoretical and practical tools for developing CSMs that balance accuracy and interpretability in a principled manner.

Quantifying the Accuracy-Interpretability Trade-Off in Concept-Based Sidechannel Models

TL;DR

The paper tackles the problem of balancing predictive accuracy with interpretability in Concept Sidechannel Models (CSMs), which extend Concept Bottleneck Models (CBMs) by using a sidechannel to boost performance. It introduces a unified probabilistic meta-model and the Sidechannel Independence Score (SIS) to quantify how much a model relies on the sidechannel, along with SIS regularization to explicitly minimize this reliance. The authors analyze expressivity in bottleneck versus default modes and demonstrate, across CelebA and MNIST-Addition, that SIS regularization yields substantial gains in representation interpretability and intervenability while preserving accuracy, with models like CMR and its enhanced variant achieving strong performance. This framework provides a principled toolkit for constructing CSMs that navigate the accuracy–interpretability trade-off and offers practical guidance for training and evaluation. The work advances both theory and practice by enabling explicit control over how much uninterpretable side information influences predictions, with broad implications for interpretable AI deployment.

Abstract

Concept Bottleneck Models (CBNMs) are deep learning models that provide interpretability by enforcing a bottleneck layer where predictions are based exclusively on human-understandable concepts. However, this constraint also restricts information flow and often results in reduced predictive accuracy. Concept Sidechannel Models (CSMs) address this limitation by introducing a sidechannel that bypasses the bottleneck and carry additional task-relevant information. While this improves accuracy, it simultaneously compromises interpretability, as predictions may rely on uninterpretable representations transmitted through sidechannels. Currently, there exists no principled technique to control this fundamental trade-off. In this paper, we close this gap. First, we present a unified probabilistic concept sidechannel meta-model that subsumes existing CSMs as special cases. Building on this framework, we introduce the Sidechannel Independence Score (SIS), a metric that quantifies a CSM's reliance on its sidechannel by contrasting predictions made with and without sidechannel information. We propose SIS regularization, which explicitly penalizes sidechannel reliance to improve interpretability. Finally, we analyze how the expressivity of the predictor and the reliance of the sidechannel jointly shape interpretability, revealing inherent trade-offs across different CSM architectures. Empirical results show that state-of-the-art CSMs, when trained solely for accuracy, exhibit low representation interpretability, and that SIS regularization substantially improves their interpretability, intervenability, and the quality of learned interpretable task predictors. Our work provides both theoretical and practical tools for developing CSMs that balance accuracy and interpretability in a principled manner.

Paper Structure

This paper contains 24 sections, 6 equations, 12 figures.

Table of Contents

  1. Introduction
  2. Background
  3. Method
  4. Representation Interpretability vs Functional Interpretability
  5. The Meta-Model of CSMs
  6. Default and Bottleneck modes for CSMs
  7. Sidechannel Independence Score
  8. Optimizing CSMs for Representation Interpretability
  9. Expressivity of CSMs in Bottleneck Mode
  10. Related Work
  11. Experiments
  12. Setup
  13. Results and Discussion
  14. Conclusion
  15. Overview of state-of-the-art CSMs
  16. ...and 9 more sections

Figures (12)

  • Figure 1: Comparison between Deep Neural Networks, (linear) Concept Bottleneck Models, and Concept Sidechannel Models. Interpretable components are green; uninterpretable components are orange. Concepts are interpretable because they are aligned to some human interpretation through direct supervision (e.g. tail).
  • Figure 2: CSM Meta-model
  • Figure 3: Comparison between CRM (left) and CMR (right). Concept scores and embeddings are denoted by deterministic delta distributions. (Un)interpretable representations and functions are (orange) green. For simplicity, we take the same neural network as sidechannel and concept predictor.
  • Figure 4: Accuracy vs representation interpretability trade-off in CSMs. Each point is a hyperparameter configuration, keeping only pareto-efficient points. Crosses are the most accurate configuration trained without SIS regularization (not in (c) and (f)). FI means "functionally interpretable".
  • Figure 5: Accuracy vs SIS for CRM on CelebA, comparing with concept usage approaches. The cross is the most accurate CRM trained without any approach.
  • ...and 7 more figures

Theorems & Definitions (5)

  • Example 3.1
  • Example 3.2: Concept Residual Model (CRM) mahinpei2021promises
  • Example 3.3: Concept Memory Reasoner (CMR) debot2024interpretable
  • Definition 3.1: Sidechannel Independence Score - SIS
  • Example 3.4