An Analysis of Concept Bottleneck Models: Measuring, Understanding, and Mitigating the Impact of Noisy Annotations

TL;DR

This work investigates how noisy concept annotations degrade the interpretability and accuracy of concept bottleneck models (CBMs) and degrades intervention effectiveness. It identifies a small susceptible subset of concepts whose corruption largely explains performance loss and introduces a two-stage mitigation: training with sharpness-aware minimization (SAM) to stabilize susceptible concepts, and inference-time uncertainty-guided intervention to correct the most uncertain concepts. The findings show SAM consistently improves concept and task accuracy across datasets and noise levels, while uncertainty-guided interventions significantly recover performance by targeting susceptible concepts, with theory linking uncertainty to susceptibility. Together, these contributions offer a principled framework for robust, interpretable CBMs in realistically noisy supervision settings, with implications for CBM variants and cross-modal applications.

Abstract

Concept bottleneck models (CBMs) ensure interpretability by decomposing predictions into human interpretable concepts. Yet the annotations used for training CBMs that enable this transparency are often noisy, and the impact of such corruption is not well understood. In this study, we present the first systematic study of noise in CBMs and show that even moderate corruption simultaneously impairs prediction performance, interpretability, and the intervention effectiveness. Our analysis identifies a susceptible subset of concepts whose accuracy declines far more than the average gap between noisy and clean supervision and whose corruption accounts for most performance loss. To mitigate this vulnerability we propose a two-stage framework. During training, sharpness-aware minimization stabilizes the learning of noise-sensitive concepts. During inference, where clean labels are unavailable, we rank concepts by predictive entropy and correct only the most uncertain ones, using uncertainty as a proxy for susceptibility. Theoretical analysis and extensive ablations elucidate why sharpness-aware training confers robustness and why uncertainty reliably identifies susceptible concepts, providing a principled basis that preserves both interpretability and resilience in the presence of noise.

Figures (20)

• Figure 1: (a) Noisy concept annotations arise easily in CBMs. (b, c) On the CUB CUB dataset, raising the noise level to 40% lowers task accuracy from 74.3% to 4.0% and reduces interpretability—assessed by the concept alignment score CEM, which measures semantic alignment between learned representations and ground truth concepts—from 84.3% to 58.4%.
• Figure 2: Impact of noise on CBMs. (a) Task accuracy degradation; (b) Source of degradation; (c) Concept accuracy degradation; (d) Decline in interpretability measured by the concept alignment score. The top row represents the CUB dataset, and the bottom row shows results for AwA2 dataset.
• Figure 3: t-SNE tsne visualization of model representations under different noise settings.
• Figure 4: t-SNE tsne visualizations of 'blue upperparts' concept embedding learnt in CUB with sample points colored red if the concept is active and blue if the concept is inactive in that sample. As noise ratio increases, the concepts are clearly entangled making concepts unreliable.
• Figure 5: (a, b, c) Impact of noise on the effectiveness of CBM interventions using Random, UCP, and CCTP strategies under varying noise levels in the CUB dataset. (d, e) Effects of performing incorrect random concept interventions at different noise rates in CUB and AwA2 dataset.

Theorems & Definitions (13)

• Definition A.1: 2-Layer Concept Bottleneck Model (CBM)
• Definition A.2: Binary Cross-Entropy (BCE) Loss
• Definition A.3: Sharpness-Aware Minimization (SAM)
• Proposition A.4
• proof
• Definition B.1: Susceptibility
• Definition B.2: Predictive Uncertainty
• Definition B.3: Concept Subsets
• Definition B.7: Kendall’s Tau Distance
• Lemma B.8: Ranking Consistency via Kendall’s Tau