Debugging Concept Bottleneck Models through Removal and Retraining

Abstract

Concept Bottleneck Models (CBMs) use a set of human-interpretable concepts to predict the final task label, enabling domain experts to not only validate the CBM's predictions, but also intervene on incorrect concepts at test time. However, these interventions fail to address systemic misalignment between the CBM and the expert's reasoning, such as when the model learns shortcuts from biased data. To address this, we present a general interpretable debugging framework for CBMs that follows a two-step process of Removal and Retraining. In the Removal step, experts use concept explanations to identify and remove any undesired concepts. In the Retraining step, we introduce CBDebug, a novel method that leverages the interpretability of CBMs as a bridge for converting concept-level user feedback into sample-level auxiliary labels. These labels are then used to apply supervised bias mitigation and targeted augmentation, reducing the model's reliance on undesired concepts. We evaluate our framework with both real and automated expert feedback, and find that CBDebug significantly outperforms prior retraining methods across multiple CBM architectures (PIP-Net, Post-hoc CBM) and benchmarks with known spurious correlations.

Figures (9)

• Figure 1: Our debugging framework for incorporating a domain expert’s knowledge into a concept bottleneck. Removal (a): The user inspects concept explanations and selects undesired concepts to remove, such as background concepts in bird classification. Retraining (b): The concept extractor and inference layer are retrained based on this feedback, updating the CBM to remove dependence on undesired concepts while maintaining reliance on task-relevant concepts.
• Figure 2: Overview of CBDebug (Concept Bottleneck Debugger), which consists of three main steps. First, the encoder $\phi$ computes the concept activations for undesired concepts in $\mathcal{C}_{spur}$ to generate the approximated auxiliary label ${\mathbf{\widehat{V}}}$. Second, permutation weighting utilizes ${\mathbf{\widehat{V}}}$ and the class label $Y$ to compute the odds of the sample being drawn from the unconfounded distribution, generating weights $\mathbf{U}$. Third, augmentation is performed on $\mathbf{X}$ based on the undesired concepts $\mathcal{C}_{spur}$ and weights $\mathbf{U}$ to generate $\mathbf{X}_{aug}$. Finally, we retrain $\{\phi, h\}$ on $(\mathbf{X}_{aug}, Y)$ weighted by $\mathbf{U}$ and return $\{\phi', h'\}$.
• Figure 3: The six most highly activated concepts for the Original model trained on Waterbirds and the model after retraining with CBDebug. CBDebug removes both concepts representing bamboo from the concept set and replaces them with more robust concepts representing bird features.
• Figure 4: Participants are shown concepts learned by the model and asked to flag those that are spurious for the classification task.
• Figure 5: Number of concepts marked as spurious during the debugging sessions for real users on PIP-Net.