LLEXICORP: End-user Explainability of Convolutional Neural Networks

TL;DR

Problem: Concept-based explanations via CRP are informative but require manual naming and narrative synthesis, limiting scalability. Approach: LLEXICORP couples CRP with a multimodal LLM using carefully separated prompts for concept naming and explanation, enabling audience-adaptive textual narratives while preserving faithfulness. Contributions: identifies CRP bottlenecks, introduces a modular CRP+LLM framework, and demonstrates a qualitative evaluation with ImageNet/VGG16 showing improved accessibility of CNN reasoning. Impact: reduces barriers to interpreting deep networks and supports broader deployment of concept-based XAI in practical vision systems.

Abstract

Convolutional neural networks (CNNs) underpin many modern computer vision systems. With applications ranging from common to critical areas, a need to explain and understand the model and its decisions (XAI) emerged. Prior works suggest that in the top layers of CNNs, the individual channels can be attributed to classifying human-understandable concepts. Concept relevance propagation (CRP) methods can backtrack predictions to these channels and find images that most activate these channels. However, current CRP workflows are largely manual: experts must inspect activation images to name the discovered concepts and must synthesize verbose explanations from relevance maps, limiting the accessibility of the explanations and their scalability. To address these issues, we introduce Large Language model EXplaIns COncept Relevance Propagation (LLEXICORP), a modular pipeline that couples CRP with a multimodal large language model. Our approach automatically assigns descriptive names to concept prototypes and generates natural-language explanations that translate quantitative relevance distributions into intuitive narratives. To ensure faithfulness, we craft prompts that teach the language model the semantics of CRP through examples and enforce a separation between naming and explanation tasks. The resulting text can be tailored to different audiences, offering low-level technical descriptions for experts and high-level summaries for non-technical stakeholders. We qualitatively evaluate our method on various images from ImageNet on a VGG16 model. Our findings suggest that integrating concept-based attribution methods with large language models can significantly lower the barrier to interpreting deep neural networks, paving the way for more transparent AI systems.

Figures (6)

• Figure 1: Exaple image of lizard from ImageNet.
• Figure 2: The activation heatmaps of the five most relevant convolutional filters, as found by CRP.
• Figure 3: Top 6 images that most activate on each of the 5 most relevant concepts, as found by CRP, together with a saliency map of activation of the said concepts.
• Figure 4: LLEXICORP generated summary for the image of the lizard and given the outputs of CRP.
• Figure 5: An example of a set of representatives of a concept with their saliency maps provided by the concept visualizer. The result of LLM's annotation is: The concept appears to be "wooden cylindrical objects arranged in parallel or patterns." This includes items like wooden sticks, matchsticks, or pipes, often aligned or forming shapes.