FastCAV: Efficient Computation of Concept Activation Vectors for Explaining Deep Neural Networks

TL;DR

This work introduces FastCAV, a novel approach that accelerates the extraction of CAVs by up to 63.6x and enables previously infeasible investigations of deep models, which is demonstrated by tracking the evolution of concepts during model training.

Abstract

Concepts such as objects, patterns, and shapes are how humans understand the world. Building on this intuition, concept-based explainability methods aim to study representations learned by deep neural networks in relation to human-understandable concepts. Here, Concept Activation Vectors (CAVs) are an important tool and can identify whether a model learned a concept or not. However, the computational cost and time requirements of existing CAV computation pose a significant challenge, particularly in large-scale, high-dimensional architectures. To address this limitation, we introduce FastCAV, a novel approach that accelerates the extraction of CAVs by up to 63.6x (on average 46.4x). We provide a theoretical foundation for our approach and give concrete assumptions under which it is equivalent to established SVM-based methods. Our empirical results demonstrate that CAVs calculated with FastCAV maintain similar performance while being more efficient and stable. In downstream applications, i.e., concept-based explanation methods, we show that FastCAV can act as a replacement leading to equivalent insights. Hence, our approach enables previously infeasible investigations of deep models, which we demonstrate by tracking the evolution of concepts during model training.

Figures (20)

• Figure 1: Comparison of computational efficiency between FastCAV and established SVM-CAV for calculating Concept Activation Vectors (CAVs) across different models. The average time to calculate a CAV for each method is plotted against the dimensionality of the activation spaces, demonstrating the significant speedup ($p < 0.01$) achieved by FastCAV.
• Figure 2: Schematic illustration of FastCAV in two dimensions. Specifically, the learned concept activation vector $v_c^l$ points from the global mean $\hat{\mu}_{D_c\cup D_r}$ towards $\hat{\mu}_{D_c}$.
• Figure 3: TCAV scores for various GoogleNet szegedy2015going layers. We compare the concepts "polka-dotted", "striped", and "zigzagged" for the class ladybug using FastCAV against SVM-CAV. We follow kim2018interpretabilityfeatureattributionquantitative and mark CAVs that are not statistically significant with "*".
• Figure 4: Most salient concepts discovered by ACE ghorbani2019towards using either our FastCAV or the established SVM-CAV. In both cases, we find the discovered patches containing stripes, which is congruent with the original observation in ghorbani2019towards.
• Figure 5: Sensitivity analysis of FastCAV to (a) the number of concept images and (b) number of resampled $D_r$. Note the differences in y-axis scales.