Rethinking Concept Bottleneck Models: From Pitfalls to Solutions

TL;DR

This work proposes an entropy-based metric to quantify the intrinsic suitability of a concept set for a given dataset and resolves the linearity problem by inserting a non-linear layer between concept activations and the classifier, which ensures that model accuracy faithfully reflects concept relevance.

Abstract

Concept Bottleneck Models (CBMs) ground predictions in human-understandable concepts but face fundamental limitations: the absence of a metric to pre-evaluate concept relevance, the "linearity problem" causing recent CBMs to bypass the concept bottleneck entirely, an accuracy gap compared to opaque models, and finally the lack of systematic study on the impact of different visual backbones and VLMs. We introduce CBM-Suite, a methodological framework to systematically addresses these challenges. First, we propose an entropy-based metric to quantify the intrinsic suitability of a concept set for a given dataset. Second, we resolve the linearity problem by inserting a non-linear layer between concept activations and the classifier, which ensures that model accuracy faithfully reflects concept relevance. Third, we narrow the accuracy gap by leveraging a distillation loss guided by a linear teacher probe. Finally, we provide comprehensive analyses on how different vision encoders, vision-language models, and concept sets interact to influence accuracy and interpretability in CBMs. Extensive evaluations show that CBM-Suite yields more accurate models and provides insights for improving concept-based interpretability.

Figures (10)

• Figure 1: Common issues among CBM approaches. (i) Models can achieve high accuracy even with irrelevant or random concepts due to concept leakage. (ii) Overlooking the importance of non-linearity reduces CBMs to mere linear probes. (iii) Introducing a concept bottleneck layer often leads to an accuracy gap compared to opaque models. (iv) Although CBMs are modular by design, evaluations do not consider different encoder choices.
• Figure 2: Linearity problem undermines interpretability. Linear CBMs attain high accuracy (85+% on ImageNet100 dataset) with random strings (left) and Roman Law terminology (right).
• Figure 3: Histogram of concept activations computed with SAIL for relevant and irrelevant concept sets on ImageNet100 and Places365, illustrating the difference in activation distributions between meaningful and random concepts.
• Figure 4: Overview of the CBM-Suite, designed to address the key pitfalls of conventional Concept Bottleneck Models. First, the most relevant concept set for the dataset is selected using the Goodness of Concepts metric. Then, a non-linear concept encoder is trained to construct a trustworthy CBM. Next, the final classifier module is trained under teacher supervision. Finally, CBM-Suite provides a comprehensive evaluation suite across different vision encoders and vision-language models.
• Figure 5: Effect of non-linear concept encoders. We report classification accuracy on ImageNet100 and Places365 using both relevant and irrelevant concept sets. Linear CBM and Non-linear CBM differ only in the concept encoder layer, i.e., the latter includes a non-linear function. Error bars represent the standard deviation over ten independent runs with random initializations.