Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bi-ICE: An Inner Interpretable Framework for Image Classification via Bi-directional Interactions between Concept and Input Embeddings

Jinyung Hong, Yearim Kim, Keun Hee Park, Sangyu Han, Nojun Kwak, Theodore P. Pavlic

TL;DR

This work introduces Bi-ICE, an inner interpretability framework for image classification that embeds human-interpretable concepts into a bi-directional memory-based interaction with input embeddings. Grounded in Marr-style multilevel analysis (computational, algorithmic, implementation), Bi-ICE uses ConceptBinding, Broadcast, and ComputeLogit to iteratively refine concept representations and localize their influence on input regions. The approach yields transparent predictions, measurable concept contributions, and patch-level localization, while maintaining or improving predictive accuracy across CIFAR100, ImageNet, and CUB200. Extensive analyses—including concept convergence, insertion/deletion tests, and a user study—demonstrate the framework’s interpretability and scalability, highlighting its potential as a principled, multilevel XAI method for visual tasks.

Abstract

Inner interpretability is a promising field aiming to uncover the internal mechanisms of AI systems through scalable, automated methods. While significant research has been conducted on large language models, limited attention has been paid to applying inner interpretability to large-scale image tasks, focusing primarily on architectural and functional levels to visualize learned concepts. In this paper, we first present a conceptual framework that supports inner interpretability and multilevel analysis for large-scale image classification tasks. Specifically, we introduce the Bi-directional Interaction between Concept and Input Embeddings (Bi-ICE) module, which facilitates interpretability across the computational, algorithmic, and implementation levels. This module enhances transparency by generating predictions based on human-understandable concepts, quantifying their contributions, and localizing them within the inputs. Finally, we showcase enhanced transparency in image classification, measuring concept contributions, and pinpointing their locations within the inputs. Our approach highlights algorithmic interpretability by demonstrating the process of concept learning and its convergence.

Bi-ICE: An Inner Interpretable Framework for Image Classification via Bi-directional Interactions between Concept and Input Embeddings

TL;DR

This work introduces Bi-ICE, an inner interpretability framework for image classification that embeds human-interpretable concepts into a bi-directional memory-based interaction with input embeddings. Grounded in Marr-style multilevel analysis (computational, algorithmic, implementation), Bi-ICE uses ConceptBinding, Broadcast, and ComputeLogit to iteratively refine concept representations and localize their influence on input regions. The approach yields transparent predictions, measurable concept contributions, and patch-level localization, while maintaining or improving predictive accuracy across CIFAR100, ImageNet, and CUB200. Extensive analyses—including concept convergence, insertion/deletion tests, and a user study—demonstrate the framework’s interpretability and scalability, highlighting its potential as a principled, multilevel XAI method for visual tasks.

Abstract

Inner interpretability is a promising field aiming to uncover the internal mechanisms of AI systems through scalable, automated methods. While significant research has been conducted on large language models, limited attention has been paid to applying inner interpretability to large-scale image tasks, focusing primarily on architectural and functional levels to visualize learned concepts. In this paper, we first present a conceptual framework that supports inner interpretability and multilevel analysis for large-scale image classification tasks. Specifically, we introduce the Bi-directional Interaction between Concept and Input Embeddings (Bi-ICE) module, which facilitates interpretability across the computational, algorithmic, and implementation levels. This module enhances transparency by generating predictions based on human-understandable concepts, quantifying their contributions, and localizing them within the inputs. Finally, we showcase enhanced transparency in image classification, measuring concept contributions, and pinpointing their locations within the inputs. Our approach highlights algorithmic interpretability by demonstrating the process of concept learning and its convergence.

Paper Structure

This paper contains 71 sections, 16 equations, 29 figures, 13 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Conceptual Multilevel Framework
  3. Computational Problems
  4. Algorithmic Descriptions
  5. Primitives & Implementation
  6. Primitive Representations & Operations
  7. Implementation Detail
  8. Concept Binding and Refinement Module TEXT
  9. Broadcast Module TEXT
  10. Logit Computational Module TEXT
  11. Training Objectives for Interpretability
  12. Explanation Loss with Concept Annotations.
  13. Sparsity Loss based on Entropy.
  14. Final Loss.
  15. Experiment
  16. ...and 56 more sections

Figures (29)

  • Figure 1: Illustration of our inner interpretability framework and our proposed module, Bi-ICE.
  • Figure 2: Primitives and Operations used as the essential components to implement our Bi-ICE module.
  • Figure 3: Implementation overview. Each component is detailed in Sec. \ref{['subsec:impl_detail']}. The training objectives are explained in Sec. \ref{['subsec:objectives']}.
  • Figure 4: t-SNEs of (a) vanilla ViT ($\boldsymbol{z}$), V-score: $0.9124$ vs. (b) Bi-ICE ($\bar{\boldsymbol{z}}$), V-score: $\boldsymbol{0.9366}$ on CUB w.o. concept annotations.
  • Figure 5: Computational-level Interpretability Analysis. Left: Concept Contributions of the class 'Brandt Cormorant', showing their impact on the model's decision. The dashed red box indicates the activated patch ($>$ 0.6). Green for spatial, and red for global. Right: Concept-Insertion and Deletion graphs of randomly selected 50 classes in CUB. $f(x)$ is the normalized test accuracy with the range $[0, 1]$.
  • ...and 24 more figures

Theorems & Definitions (5)

  • Definition 1: Concept
  • Definition 2: Concept Vector
  • Definition 3: Concept Score
  • Definition 4: Image-patch-specific Concept Decomposition
  • Definition 5: Classification with Concept Decomposition