Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Error Analysis of Shapley Value-Based Model Explanations: An Informative Perspective

Ningsheng Zhao, Jia Yuan Yu, Krzysztof Dzieciolowski, Trang Bui

TL;DR

Shapley-value explanations are prone to bias when data sparsity and modeling assumptions distort the conditional removal used to compute attributions. The authors propose a unified error analysis framework that splits explanation errors into observation bias and structural bias and define over-informative and under-informative explanations, plus a metric to quantify distribution drift between the true data distribution and drifted approximations. They show that assumption-based SVA methods can yield under-informative attributions due to distribution drift, while excessive data smoothing can induce observation bias in sparse regions. Experiments on Bike Sharing and Census Income datasets corroborate the theory and demonstrate how different SVA strategies can be over- or under-informative, guiding the development of more robust, less error-prone explainers.

Abstract

Shapley value attribution (SVA) is an increasingly popular explainable AI (XAI) method, which quantifies the contribution of each feature to the model's output. However, recent work has shown that most existing methods to implement SVAs have some drawbacks, resulting in biased or unreliable explanations that fail to correctly capture the true intrinsic relationships between features and model outputs. Moreover, the mechanism and consequences of these drawbacks have not been discussed systematically. In this paper, we propose a novel error theoretical analysis framework, in which the explanation errors of SVAs are decomposed into two components: observation bias and structural bias. We further clarify the underlying causes of these two biases and demonstrate that there is a trade-off between them. Based on this error analysis framework, we develop two novel concepts: over-informative and underinformative explanations. We demonstrate how these concepts can be effectively used to understand potential errors of existing SVA methods. In particular, for the widely deployed assumption-based SVAs, we find that they can easily be under-informative due to the distribution drift caused by distributional assumptions. We propose a measurement tool to quantify such a distribution drift. Finally, our experiments illustrate how different existing SVA methods can be over- or under-informative. Our work sheds light on how errors incur in the estimation of SVAs and encourages new less error-prone methods.

Error Analysis of Shapley Value-Based Model Explanations: An Informative Perspective

TL;DR

Shapley-value explanations are prone to bias when data sparsity and modeling assumptions distort the conditional removal used to compute attributions. The authors propose a unified error analysis framework that splits explanation errors into observation bias and structural bias and define over-informative and under-informative explanations, plus a metric to quantify distribution drift between the true data distribution and drifted approximations. They show that assumption-based SVA methods can yield under-informative attributions due to distribution drift, while excessive data smoothing can induce observation bias in sparse regions. Experiments on Bike Sharing and Census Income datasets corroborate the theory and demonstrate how different SVA strategies can be over- or under-informative, guiding the development of more robust, less error-prone explainers.

Abstract

Shapley value attribution (SVA) is an increasingly popular explainable AI (XAI) method, which quantifies the contribution of each feature to the model's output. However, recent work has shown that most existing methods to implement SVAs have some drawbacks, resulting in biased or unreliable explanations that fail to correctly capture the true intrinsic relationships between features and model outputs. Moreover, the mechanism and consequences of these drawbacks have not been discussed systematically. In this paper, we propose a novel error theoretical analysis framework, in which the explanation errors of SVAs are decomposed into two components: observation bias and structural bias. We further clarify the underlying causes of these two biases and demonstrate that there is a trade-off between them. Based on this error analysis framework, we develop two novel concepts: over-informative and underinformative explanations. We demonstrate how these concepts can be effectively used to understand potential errors of existing SVA methods. In particular, for the widely deployed assumption-based SVAs, we find that they can easily be under-informative due to the distribution drift caused by distributional assumptions. We propose a measurement tool to quantify such a distribution drift. Finally, our experiments illustrate how different existing SVA methods can be over- or under-informative. Our work sheds light on how errors incur in the estimation of SVAs and encourages new less error-prone methods.
Paper Structure (8 sections, 4 equations, 1 figure)

This paper contains 8 sections, 4 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Background
  3. Notation
  4. Informative SVA
  5. Observation Bias & Structural Bias Trade-Off
  6. Overfitting and Underfitting of the RF
  7. Explanation Error Decomposition
  8. Over-informative Explanation

Figures (1)

  • Figure 1: An illustration of the trade-off between observation bias and structural bias. On the one hand, to reduce observation bias, it is necessary to alleviate the data sparsity, which requires us to decrease the structural complexity of the conditional RF approximation. However, this simplification of structural complexity might concurrently lead to an increase in structural bias. On the other hand, to reduce structural bias, we may need to increase the structural complexity, which inevitably entails an aggravation of the data sparsity, consequently increasing the observation bias.