Knowledge graphs for empirical concept retrieval

TL;DR

The paper addresses the challenge of defining data-driven concepts for concept-based XAI by leveraging formal knowledge graphs (WordNet, Wikidata, ConceptNet) in an interactive workflow that retrieves data from Wikimedia Commons and Wikipedia. It evaluates how Concept Activation Vectors (CAVs) and Concept Activation Regions (CARs) perform when concept data are KG-derived, demonstrating robustness to negative-sample variations and some distribution shifts, while also showing alignment between model representations and human knowledge graph structures. The results indicate KG-based concepts can yield robust, accurate explanations and reveal meaningful human-machine alignment at concept and sub-concept levels. This suggests KG-driven concept definitions can enhance personalized, interpretable AI in text and image domains.

Abstract

Concept-based explainable AI is promising as a tool to improve the understanding of complex models at the premises of a given user, viz.\ as a tool for personalized explainability. An important class of concept-based explainability methods is constructed with empirically defined concepts, indirectly defined through a set of positive and negative examples, as in the TCAV approach (Kim et al., 2018). While it is appealing to the user to avoid formal definitions of concepts and their operationalization, it can be challenging to establish relevant concept datasets. Here, we address this challenge using general knowledge graphs (such as, e.g., Wikidata or WordNet) for comprehensive concept definition and present a workflow for user-driven data collection in both text and image domains. The concepts derived from knowledge graphs are defined interactively, providing an opportunity for personalization and ensuring that the concepts reflect the user's intentions. We test the retrieved concept datasets on two concept-based explainability methods, namely concept activation vectors (CAVs) and concept activation regions (CARs) (Crabbe and van der Schaar, 2022). We show that CAVs and CARs based on these empirical concept datasets provide robust and accurate explanations. Importantly, we also find good alignment between the models' representations of concepts and the structure of knowledge graphs, i.e., human representations. This supports our conclusion that knowledge graph-based concepts are relevant for XAI.

Figures (9)

• Figure 1: Main research questions and workflow: 1) Can knowledge graphs be used to generate data-driven concepts? 2) How robust are the explanations (here CAVs) provided by such concepts to variations in the dataset? 3) Are internal representations aligned with human knowledge?
• Figure 2: Decision tree for choosing an appropriate knowledge graph based on the individual needs and wants. This is to give a general overview, individual decisions might require more consideration or different KGs. We will use WordNet and Wikidata (indicated by the orange box) for our main experiments.
• Figure 3: Extracted KGs from WordNet for the main concepts sport (orange), fruit (purple) and motor vehicle (blue) -- only sub-concepts with more than 50 images available are shown. The KGs for text are slightly different since there we filter out sub-concepts with fewer than 50 sentences.
• Figure 4: Examples of images for the concept "banana" and "cat" from Wikimedia Commons (CC0-License).
• Figure 5: Accuracy of CAVs and CARs trained on both Wikimedia and Pascal VOC data with 200 concept images (using 10-fold cross-validation) for data2vec. Baseline performance achieved by random guessing amounts to 50%.