Incremental XAI: Memorable Understanding of AI with Incremental Explanations

TL;DR

This work addresses the challenge that explainable AI explanations must be faithful yet memorable. It introduces Incremental XAI, a framework that partitions explanations into a base global model plus additive incremental factors for outlier subspaces, implemented via a linear-model-tree and sparse regularization. Through modeling, formative, and summative studies on housing prices, heart disease risk, and MPG prediction, the authors show that Incremental explanations achieve better memorability and comparable faithfulness to Subglobal explanations, while preserving reading efficiency similar to Global explanations. The approach promises improved usability and generalizes to non-linear models and other explanation formats, with implications for designing human-centered, cumulative XAI systems.

Abstract

Many explainable AI (XAI) techniques strive for interpretability by providing concise salient information, such as sparse linear factors. However, users either only see inaccurate global explanations, or highly-varying local explanations. We propose to provide more detailed explanations by leveraging the human cognitive capacity to accumulate knowledge by incrementally receiving more details. Focusing on linear factor explanations (factors $\times$ values = outcome), we introduce Incremental XAI to automatically partition explanations for general and atypical instances by providing Base + Incremental factors to help users read and remember more faithful explanations. Memorability is improved by reusing base factors and reducing the number of factors shown in atypical cases. In modeling, formative, and summative user studies, we evaluated the faithfulness, memorability and understandability of Incremental XAI against baseline explanation methods. This work contributes towards more usable explanation that users can better ingrain to facilitate intuitive engagement with AI.

Figures (32)

• Figure 1: Conceptual examples of XAI types with univariate (1D) data shown for simplicity; see Fig. \ref{['fig:xai_types_2d']} for 2D multivariate examples with real data. a) Original AI System predicts output $\hat{y}$ non-linearly with respect to attribute $x_r$. b) Global explainer that approximates $\hat{y}$ with a linear equation $\tilde{y}_g \propto x_r$. c) Subglobal explainer increases faithfulness by segmenting along $x_r$ to provide multiple linear explanations $\tilde{y}_{s_1} \propto x_r, x_r < \chi_r$ and $\tilde{y}_{s_2} \propto x_r, x_r \geq \chi_r$. d) Incremental explainer that is similar to Subglobal, but first explains with a linear model $\tilde{y}_{i_0} \propto x_r$ the contiguous majority of instances (in this case, $x_r < \chi_r$), then explains outlier instances ($x_r \geq \chi_r$) with an additive linear model $\tilde{y}_{i_0} + \Delta\tilde{y}_{\Delta i}$. e) Local explanation explains each instance with a linear equation $\tilde{y}_l \propto x_r$ based on neighboring instances. Multiple local explanations are needed to represent the full input space.
• Figure 2: User interface (UI) of AI System with Global explanation showing: 1) attributes used for prediction, 2) their values $x^{(r)}$ for the given instance, 3) factors $w_r$ that the explainer multiplies with values, 4) partial contributions $\tilde{y}_i = w^{(r)} x^{(r)}$ of each attribute, 5) output estimation $\tilde{y} = \sum_{r}{\tilde{y}^{(r)}}$ from the AI Explainer, 6) prediction $\hat{y}$ from the AI System, with inequality indication ($<$ in this case), and 7) indicator of how different the AI Explainer estimation is from the AI System prediction. Factors are the same for all instances and do no change. Different information may be hidden under various test conditions.
• Figure 3: User interface (UI) of Subglobal explanations for a typical instance (top), and an outlier instance (bottom). Factors are different for each subspace but apply in a fixed way to any instance in each subspace. For example, while small houses with Living Area < 2.5 ksqft have each bathroom being worth $16k, larger houses have much costlier bathrooms at $57k.
• Figure 4: User interface (UI) of Incremental explanation for an instance in the typical subspace with Living Area < 2.5 (top), and an outlier instance in the minority subspace with Living Area $\geq$ 2.5 ksqft (bottom). Factors are different for each subspace to fit them accurately. Unlike Subglobal explanations, an additional column (3b) is used to show how factors are incrementally different for the outlier cases. The main factors (3) are the same for both subspaces. For example, while smaller houses have a modest rate of price increase per living area ($95k/ksqft), larger houses have a rate that is $120k/ksqft higher ($215k/ksqft).
• Figure 5: User interface (UI) of the Local explanation of an instance. Factors are specific to this and similar instances, and will be different for other instances. For example, for houses similar to the one shown, each increase in Grade decreases the house price by $7.1k, but this may not be the case for other houses that have very different attributes.