Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Approximate Shapley value estimation using sampling without replacement and variance estimation via the new Symmetric bootstrap and the Doubled half bootstrap

Fredrik Lohne Aanes

TL;DR

This work integrates finite-population sampling theory with KernelSHAP to enable sampling without replacement for coalition estimation, aided by Wallenius' noncentral hypergeometric distribution to allocate counts across coalition groups. It introduces two variance estimators—the Symmetric bootstrap and the Doubled half bootstrap—and demonstrates their effectiveness through two simulation studies, finding performance on par with the shapr implementation that relies on with-replacement sampling. A Horvitz-Thompson–style estimator is discussed but deemed invalid in this context, while carefully derived bootstrap conditions ensure valid variance estimation under finite-population constraints. Overall, the proposed method delivers comparable Shapley value estimation accuracy to KernelSHAP while providing straightforward, fast, and statistically grounded variance estimates suitable for finite populations.

Abstract

In this paper I consider improving the KernelSHAP algorithm. I suggest to use the Wallenius' noncentral hypergeometric distribution for sampling the number of coalitions and perform sampling without replacement, so that the KernelSHAP estimation framework is improved further. I also introduce the Symmetric bootstrap to calculate the standard deviations and also use the Doubled half bootstrap method to compare the performance. The new bootstrap algorithm performs better or equally well in the two simulation studies performed in this paper. The new KernelSHAP algorithm performs similarly as the improved KernelSHAP method in the state-of-the-art R-package shapr, which samples coalitions with replacement in one of the options

Approximate Shapley value estimation using sampling without replacement and variance estimation via the new Symmetric bootstrap and the Doubled half bootstrap

TL;DR

This work integrates finite-population sampling theory with KernelSHAP to enable sampling without replacement for coalition estimation, aided by Wallenius' noncentral hypergeometric distribution to allocate counts across coalition groups. It introduces two variance estimators—the Symmetric bootstrap and the Doubled half bootstrap—and demonstrates their effectiveness through two simulation studies, finding performance on par with the shapr implementation that relies on with-replacement sampling. A Horvitz-Thompson–style estimator is discussed but deemed invalid in this context, while carefully derived bootstrap conditions ensure valid variance estimation under finite-population constraints. Overall, the proposed method delivers comparable Shapley value estimation accuracy to KernelSHAP while providing straightforward, fast, and statistically grounded variance estimates suitable for finite populations.

Abstract

In this paper I consider improving the KernelSHAP algorithm. I suggest to use the Wallenius' noncentral hypergeometric distribution for sampling the number of coalitions and perform sampling without replacement, so that the KernelSHAP estimation framework is improved further. I also introduce the Symmetric bootstrap to calculate the standard deviations and also use the Doubled half bootstrap method to compare the performance. The new bootstrap algorithm performs better or equally well in the two simulation studies performed in this paper. The new KernelSHAP algorithm performs similarly as the improved KernelSHAP method in the state-of-the-art R-package shapr, which samples coalitions with replacement in one of the options
Paper Structure (18 sections, 18 equations, 8 figures)

This paper contains 18 sections, 18 equations, 8 figures.

Table of Contents

  1. Introduction
  2. Finite population/Survey theory
  3. Brief overview of Shapley values used to explain individual model predictions
  4. Combining finite population sampling theory with Shapley value estimation theory
  5. Comparison with KernelSHAP
  6. Variance estimation of Shapley values
  7. Necessary conditions for validity of bootstrap methods for finite populations
  8. Horvitz-Thompson like estimator
  9. The Doubled half bootstrap
  10. The Symmetric bootstrap method
  11. Verifying the correctness of the new method
  12. Non-integer $n_0$, $n_1$ or $n_2$
  13. Comparison with other methods
  14. Variance estimation in shapr
  15. Simulation studies
  16. ...and 3 more sections

Figures (8)

  • Figure 1: Small study: The bootstrap estimates of the standard deviations obtained using shapr are plotted against the standard deviation based on running shapr 300 times, finding the mean and taking the standard deviation. Both estimates are based on the same runs of shapr. There are more details provided in the text. Along the x-axis is the bootstrap estimates while along the y-axis is the true standard deviations.
  • Figure 2: Small study: The bootstrap estimates of the standard deviations obtained using the new method for the sampling of coalitions combined with the Antal-Tillé bootstrap method are plotted against the standard deviation based on running the new algorithm 300 times, finding the mean and taking the standard deviation. Both estimates are based on the same runs of the new algorithm. More details are given in the text. Along the x-axis is the bootstrap estimates while along the y-axis is the true standard deviations.
  • Figure 3: Small study: The bootstrap estimates of the standard deviations based on the Symmetric bootstrap obtained using the new method for sampling coalitions are plotted against the standard deviation based on running the new algorithm 300 times, finding the mean in each run and taking the standard deviation. Both estimates are based on the same runs of the new algorithm. More details are provided in the text. Along the x-axis is the bootstrap estimates while along the y-axis is the true standard deviations.
  • Figure 4: Small study: The estimates of the standard deviations based on resampling 300 coalitions using either the new or the shapr-method are plotted against each other. Along the x-axis is the estimates using the new method, while along the y-axis is the estimates from using the shapr-package.
  • Figure 5: Large study: Bootstrap estimates of the standard deviations obtained either through Antal-Tillé (x-axis) or the Symmetric bootstrap (y-axis.)
  • ...and 3 more figures