Exploiting Interpretable Capabilities with Concept-Enhanced Diffusion and Prototype Networks

TL;DR

This work proposes Concept-Guided Conditional Diffusion, which can generate visual representations of concepts, and Concept-Guided Prototype Networks, which can create a concept prototype dataset and leverage it to perform interpretable concept prediction.

Abstract

Concept-based machine learning methods have increasingly gained importance due to the growing interest in making neural networks interpretable. However, concept annotations are generally challenging to obtain, making it crucial to leverage all their prior knowledge. By creating concept-enriched models that incorporate concept information into existing architectures, we exploit their interpretable capabilities to the fullest extent. In particular, we propose Concept-Guided Conditional Diffusion, which can generate visual representations of concepts, and Concept-Guided Prototype Networks, which can create a concept prototype dataset and leverage it to perform interpretable concept prediction. These results open up new lines of research by exploiting pre-existing information in the quest for rendering machine learning more human-understandable.

Figures (23)

• Figure 1: Overview of the introduced concept-enhanced models. (a) Example of explored CUB and AWA2 datasets. (b, c) Summary of the methodology of Concept-Guided Conditional Diffusion and Prototype Networks. The resulting generations and prototype visualizations are shown in purple.
• Figure 1: Accuracy results for both Concept-Guided Prototype Networks with varying base architectures, highlighting the best-performing models. Results are compared with the black-box oracle.
• Figure 2: Concept visualizations (positive and negative) generated with Concept-Guided Conditional Diffusion for both datasets and different combinations of concepts. Each row corresponds to one of the embedding types described (positive, opposite, double).
• Figure 3: Concept prototype visualizations for Concept-Guided ProtoPNet (left) and ProtoPools (right). First row shows the activation map over the original image and the second row shows the corresponding prototype.
• Figure 4: Schematic representation of the three types of embedding proposed for the Concept-Guided Conditional Diffusion model.