Transferable XAI: Relating Understanding Across Domains with Explanation Transfer

TL;DR

Compared to single-domain and domain-independent explanations, Transferable XAI was the most helpful for understanding the second domain, leading to the best decision faithfulness, factor recall, and ability to relate explanations between domains.

Abstract

Current Explainable AI (XAI) focuses on explaining a single application, but when encountering related applications, users may rely on their prior understanding from previous explanations. This leads to either overgeneralization and AI overreliance, or burdensome independent memorization. Indeed, related decision tasks can share explanatory factors, but with some notable differences; e.g., body mass index (BMI) affects the risks for heart disease and diabetes at the same rate, but chest pain is more indicative of heart disease. Similarly, models using different attributes for the same task still share signals; e.g., temperature and pressure affect air pollution but in opposite directions due to the ideal gas law. Leveraging transfer of learning, we propose Transferable XAI to enable users to transfer understanding across related domains by explaining the relationship between domain explanations using a general affine transformation framework applied to linear factor explanations. The framework supports explanation transfer across various domain types: translation for data subspace (subsuming prior work on Incremental XAI), scaling for decision task, and mapping for attributes. Focusing on task and attributes domain types, in formative and summative user studies, we investigated how well participants could understand AI decisions from one domain to another. Compared to single-domain and domain-independent explanations, Transferable XAI was the most helpful for understanding the second domain, leading to the best decision faithfulness, factor recall, and ability to relate explanations between domains. This framework contributes to improving the reusability of explanations across related AI applications by explaining factor relationships between subspaces, tasks, and attributes.

Figures (33)

• Figure 1: Concept of Transferable XAI for cross-domain understanding. a) Original Domain: Initial understanding of AI decisions. b) Target Domain: Understanding goal for the second domain. c) Mismatched Domains: Applying original understanding to the target domain results in knowledge gaps (gray) and potential errors. d) Concatenated Domains: Learning both domains independently is cognitively demanding and covers shared knowledge redundantly (dark blue). e) Transferred Across Domains: Leveraging original understanding, the user can transfer understanding across domains for efficient understanding of the target domain.
• Figure 2: Conceptual example of affine transformations across different domain types, shown with 2-dimensional data (two attributes) for simplicity. See technical details in Section \ref{['sec:affine-transformation']}.
• Figure 3: Overview of evaluations across two experiments, Experiment I: Task Transfer and Experiment II: Attributes Transfer. a) Both experiments share the same Base XAI with tabular UI for instances of the Original domain. But for Attributes Transfer, we omit Original XAI for Target domain XAI Types. b) Both experiments share the same experiment procedure and c) in each step, mostly share the same dependent variables. But for Task Transfer, we measure Factor Scale recall, and for Attributes Transfer, we measure Factor Mapping recall. d) For forward simulation trials, there are four between-subjects arrangements that participants see based on assigned Application, Domain Scenario. This determines which are their Original or Target domains. Arrangements shown for Task Transfer; attribute sets vary for Attributes Transfer. Note that the experiment design is the same for the formative and the summative user studies in each experiment.
• Figure 4: User interface (UI) of AI System with Base XAI of the Original domain showing: 1) attributes used in the prediction task, 2) instance relative values, computed as the actual value minus the average value for each attribute, i.e., $x^{(r)}$- $\bar{x}^{(r)}$, 3) factors $w^{(r)}$ of each attributes, 4) partial contributions $w^{(r)}x^{(r)}$ of each attributes, 5) estimation $\tilde{y} = \sum_r \tilde{y}^{(r)}$ from the AI Explainer, 6) prediction $\hat{y}$ from the AI System, 7) indicator of how different the AI Explainer estimation is from the AI System.
• Figure 5: Forward simulation task in summative user study main trial in three-page arrangement for each trial: a) Test page for user to estimate XAI and AI predictions, b) Explanation page to read XAI and update estimation of AI prediction, c) Learning page to review estimations with respect to actual XAI and AI predictions. Example shown is for an instance in the Target domain for the Task transfer experiment.