CoE: Chain-of-Explanation via Automatic Visual Concept Circuit Description and Polysemanticity Quantification

TL;DR

This paper tackles the challenge of explainability in deep vision models by addressing both global and local levels with a novel Chain-of-Explanation (CoE) framework. It introduces Automatic Concept Decoding and Description (ACD) to build a global linguistic concept dataset, followed by Concept Polysemanticity Disentanglement and Filtering (CPDF) to decompose polysemantic Visual Concepts into interpretable atoms, quantified by a Concept Polysemanticity Entropy (CPE). A local explanation chain is then constructed to generate linguistically grounded explanations via LLMs, enabling CoE-based local reasoning akin to Chain-of-Thought. Quantitative and human evaluations show that CoE, especially with CPDF and filtering, substantially improves explainability scores (average absolute improvement around 36%), and CPE correlates with perceived polysemanticity, supporting more reliable and scalable XAI for models like ResNet and CLIP.

Abstract

Explainability is a critical factor influencing the wide deployment of deep vision models (DVMs). Concept-based post-hoc explanation methods can provide both global and local insights into model decisions. However, current methods in this field face challenges in that they are inflexible to automatically construct accurate and sufficient linguistic explanations for global concepts and local circuits. Particularly, the intrinsic polysemanticity in semantic Visual Concepts (VCs) impedes the interpretability of concepts and DVMs, which is underestimated severely. In this paper, we propose a Chain-of-Explanation (CoE) approach to address these issues. Specifically, CoE automates the decoding and description of VCs to construct global concept explanation datasets. Further, to alleviate the effect of polysemanticity on model explainability, we design a concept polysemanticity disentanglement and filtering mechanism to distinguish the most contextually relevant concept atoms. Besides, a Concept Polysemanticity Entropy (CPE), as a measure of model interpretability, is formulated to quantify the degree of concept uncertainty. The modeling of deterministic concepts is upgraded to uncertain concept atom distributions. Finally, CoE automatically enables linguistic local explanations of the decision-making process of DVMs by tracing the concept circuit. GPT-4o and human-based experiments demonstrate the effectiveness of CPE and the superiority of CoE, achieving an average absolute improvement of 36% in terms of explainability scores.

Figures (10)

• Figure 1: Illustration of the integrated process of concept-based post-hoc XAI methods lrpsurvey, along with two challenges they face.
• Figure 2: Outline of the Chain-of-Explanation. It consists of three steps (i.e., ACD, CPDF, and local explanation). Each part utilizes a set of channels $\mathcal{C}$ from $\mathcal{M}$. In the concept circuit, $\mathcal{P}$, lines of different depths and colors represent the magnitude of the relevance.
• Figure 3: Illustrations of CPE scores at 3 levels of granularity. In (a), the channel indices of each layer are max normalized to 1.
• Figure S4: Examples of two versions of the disentangled concept atom probability distributions. (a) shows the naive version, while (b) represents the clustered one. The channel showed here is number 163 of the output layer of Stage 4 of a ResNet152 model.
• Figure S5: Averaged quantity of disentangled concept atoms. The average term is the averaged quantity of 4 stages.