Evaluate with the Inverse: Efficient Approximation of Latent Explanation Quality Distribution

TL;DR

The paper tackles the problem of evaluating explanations when ground-truth explanations are unavailable and standard baselines are insufficient for ranking quality. It introduces Quality Gap Estimator (QGE), defined as $QGE = \Psi(\mathbf{e}, \mathbf{x}, f, \hat{y}) - \Psi(\mathbf{e}^{inv}, \mathbf{x}, f, \hat{y})$, which compares an explanation to its inverse, enabling relative quality judgments with only two evaluations of the quality function. Extensive experiments across datasets (e.g., MNIST, CIFAR, ImageNet, 20Newsgroups) and metrics (faithfulness, localization, robustness) show that QGE improves rank reliability (e.g., Kendall’s $\tau$, Spearman $\rho$) and often outperforms random-baseline methods while reducing computational costs. The work provides practical tooling via the Quantus package, supporting adoption by practitioners and paving the way for more informative XAI evaluation and inspection of model behavior.

Abstract

Obtaining high-quality explanations of a model's output enables developers to identify and correct biases, align the system's behavior with human values, and ensure ethical compliance. Explainable Artificial Intelligence (XAI) practitioners rely on specific measures to gauge the quality of such explanations. These measures assess key attributes, such as how closely an explanation aligns with a model's decision process (faithfulness), how accurately it pinpoints the relevant input features (localization), and its consistency across different cases (robustness). Despite providing valuable information, these measures do not fully address a critical practitioner's concern: how does the quality of a given explanation compare to other potential explanations? Traditionally, the quality of an explanation has been assessed by comparing it to a randomly generated counterpart. This paper introduces an alternative: the Quality Gap Estimate (QGE). The QGE method offers a direct comparison to what can be viewed as the `inverse' explanation, one that conceptually represents the antithesis of the original explanation. Our extensive testing across multiple model architectures, datasets, and established quality metrics demonstrates that the QGE method is superior to the traditional approach. Furthermore, we show that QGE enhances the statistical reliability of these quality assessments. This advance represents a significant step toward a more insightful evaluation of explanations that enables a more effective inspection of a model's behavior.

Figures (7)

• Figure 1: (Step 1). The usual XAI pipeline allows the user to obtain an explanation $e$ for a prediction $\hat{y}$ using any explanation method. This is demonstrated for two distinct inputs ($\mathbf{x_0}$ and $\mathbf{x_1}$), producing predictions $\hat{y}_0$ and $\hat{y}1$, and explanations $\mathbf{e_0}$ and $\mathbf{e_1}$, respectively. (Step 2). To assess the quality of explanation $\mathbf{e}$ for prediction $\hat{y}$, the user computes a quality measure $q$. In this example, we use the area under the Pixel-Flipping curve, though the method can work with any attribution-based quality measure. (Problem) Despite both input/explanation pairs registering identical $q$ values, it remains unknown to the user that $\mathbf{e_0}$ has higher quality than most explanations for the first prediction, while $\mathbf{e_1}$ has average quality compared to other explanations for the second prediction, as shown by their histograms. (Solution) To allow the user to effectively gauge the relative quality of explanation $\mathbf{e}$ against alternative explanations $\mathbf{e'}$, we introduce $\texttt{QGE}$, which measures the difference between the quality of $\mathbf{e}$ and the quality of $\mathbf{e}_{inv}$ (a rearrangement of $\mathbf{e}$ ranking features in reverse order). This comparative quality metric does not require costly sampling of the $q$ distribution. Although both explanations have equivalent $q$ values, using QGE, the user can discern that $\mathbf{e_0}$ is a high-quality explanation for the first prediction, while $\mathbf{e_1}$ is merely average for the second one. The user may then seek a better explanation for the second prediction.
• Figure 2: Kendall's $\tau$ for the Avila and Glass datasets. The blue line indicates the average $\tau_{q,\texttt{QRAND}_{\text{K}}}$ for each value of $K$ over 5 different inputs, with the shaded area showing the average $\pm\sigma$. The orange line records the average $\tau_{q,\texttt{QGE}}$, with dashed lines representing the average $\pm\sigma$.)
• Figure 3: Kendall's $\tau$ for explanations of a given level of exceptionality. The blue line represents the average correlation $\tau_{q,\texttt{QRAND}_{\text{K}}}$ for each $K$ value across 10 different inputs, with the shaded area indicating the average$\pm\sigma$. The orange line shows the average correlation $\tau_{q,\texttt{QGE}}$, with dashed lines marking the average $\pm\sigma$.
• Figure 4: Density histogram of QGE for every possible explanation of 5 different input samples for datasets Avila and Glass.
• Figure 5: Spearman correlation of $qt$ with the original $q$ for the Avila and Glass datasets. The blue line indicates the average correlation $\rho_{q,\texttt{QRAND}_{\text{K}}}$ for each value of $K$ over 5 different inputs. The shaded area shows the average $\pm\sigma$. The orange line records the average correlation $\rho_{q,\texttt{QGE}}$ with dashed lines representing the average $\pm\sigma$.