Comparables XAI: Faithful Example-based AI Explanations with Counterfactual Trace Adjustments

TL;DR

This work contributes a new analytical basis for using example-based explanations to improve user understanding of AI decisions by proposing Comparables XAI for relatable example-based explanations of AI with Trace adjustments that trace counterfactual changes from each Comparable to the Subject, one attribute at a time, monotonically along the AI feature space.

Abstract

Explaining with examples is an intuitive way to justify AI decisions. However, it is challenging to understand how a decision value should change relative to the examples with many features differing by large amounts. We draw from real estate valuation that uses Comparables-examples with known values for comparison. Estimates are made more accurate by hypothetically adjusting the attributes of each Comparable and correspondingly changing the value based on factors. We propose Comparables XAI for relatable example-based explanations of AI with Trace adjustments that trace counterfactual changes from each Comparable to the Subject, one attribute at a time, monotonically along the AI feature space. In modelling and user studies, Trace-adjusted Comparables achieved the highest XAI faithfulness and precision, user accuracy, and narrowest uncertainty bounds compared to linear regression, linearly adjusted Comparables, or unadjusted Comparables. This work contributes a new analytical basis for using example-based explanations to improve user understanding of AI decisions.

Figures (30)

• Figure 1: Trend-based explanations along the accuracy-interpretability trade-off. Linear Trends provide simple, highly interpretable relationships, but least accurate or faithful. Non-Linear Trends are more accurate, but more complex and less interpretable. Counterfactual Traces strike a balance with a step-by-step, incremental trace of non-linear trends.
• Figure 2: Conceptual examples of XAI types with univariate (1D) data shown for simplicity. The background gray curve indicates the original AI system prediction $\hat{y}$, which is nonlinear with respect to input $x$. Given a subject case $s$ to be explained: a) Weighted averages the outcomes of Comparables ($c_1$, $c_2$, etc.), yielding an estimate $\bar{s}$ (dashed line), while the linear regression fits a single linear model across all Comparables (solid line), producing a prediction $\breve{s}$. b) Comparables with Linear Adjustments fits a local linear regression between each Comparable and the subject case $s$, producing adjusted estimates such as $\check{s}_1$ and $\check{s}_2$. c) Comparables with Trace Adjustments starts from each Comparable and stepwise adjustments are traced along the AI's underlying function ($\hat{y}$), yielding more consistent adjusted estimates $\tilde{s}_1$ and $\tilde{s}_2$ that more faithfully follow the nonlinear shape of the AI system.
• Figure 3: Trade-off between accuracy and interpretability for different example-based explanations: 1) Comparables Only averaging is simplest and most interpretable, 2) Comparables w/ Linear Regression has more sophisticated aggregating which improves accuracy but at the cost of interpretability, 3) Comparables w/ Linear Adjustments further trade-off accuracy for interpretability, and 4) Comparables w/ Trace Adjustments leverages non-linearity for the highest accuracy, but in steps to retain interpretability. The grey solid line shows the Pareto front of similarly performing methods. Positions are illustrative only rather than factual.
• Figure 4: Modeling study results for different XAI types in the House Price domain, varying the Number of Comparables (a--c) and the Average Distance of Comparables (d--f), distance was computed using the Manhattan metric with standardized features. Increasing the Number of Comparables reduces a) prediction error and b) unfaithfulness across all methods, while c) uncertainty bounds become broader. Increasing the Average Distance of Comparables worsens all metrics (d--f). Trends are smoothed with a cubic spline (smoothing parameter $\lambda=1000$). Across all conditions, Comparables w/ Trace Adjustments consistently yield lower prediction error (a, d), lower unfaithfulness (b, e), and narrower uncertainty bounds (c, f).
• Figure 5: Comparables Only and Comparables w/ Linear Regression: a) The shared interface used by both conditions. (1) rows listing all attributes; (2) a column displaying the Subject property; (5) columns showing the Subject's Comparables, annotated with similarity scores (6); (3) a row showing the Actual Price, where the red $\approx$ (3a) indicates an estimated value for the Subject; (4) shows the AI Prediction and the corresponding Prediction Error (4a) compared to Actual Price. b) The tooltip for Comparables w/ Linear Regression when hovering over (3a), presenting regression coefficients and the regression-based estimate. c) The tooltip for Comparables Only when hovering over (3a), showing a weighted-average estimate based on similarity scores.