Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient and Accurate Explanation Estimation with Distribution Compression

Hubert Baniecki, Giuseppe Casalicchio, Bernd Bischl, Przemyslaw Biecek

TL;DR

The paper tackles the challenge of efficiently and accurately estimating post-hoc explanations at scale by showing that standard i.i.d. sampling introduces non-negligible approximation error. It introduces Compress Then Explain (CTE), which replaces background sampling with distribution compression via kernel thinning to form a small, representative coreset that closely matches the marginal data distribution. The authors prove bounds showing that explanation error is controlled by the maximum mean discrepancy (MMD) between the compressed and full distributions and demonstrate that CTE reduces sample complexity by 2–3x while improving stability across multiple explanation methods (e.g., SHAP, SAGE, Expected Gradients) on diverse datasets. The approach is a lightweight plug-in with negligible overhead that can be applied to a broad class of explanations, enabling more reliable and scalable explainability in real-world ML systems.

Abstract

We discover a theoretical connection between explanation estimation and distribution compression that significantly improves the approximation of feature attributions, importance, and effects. While the exact computation of various machine learning explanations requires numerous model inferences and becomes impractical, the computational cost of approximation increases with an ever-increasing size of data and model parameters. We show that the standard i.i.d. sampling used in a broad spectrum of algorithms for post-hoc explanation leads to an approximation error worthy of improvement. To this end, we introduce Compress Then Explain (CTE), a new paradigm of sample-efficient explainability. It relies on distribution compression through kernel thinning to obtain a data sample that best approximates its marginal distribution. CTE significantly improves the accuracy and stability of explanation estimation with negligible computational overhead. It often achieves an on-par explanation approximation error 2-3x faster by using fewer samples, i.e. requiring 2-3x fewer model evaluations. CTE is a simple, yet powerful, plug-in for any explanation method that now relies on i.i.d. sampling.

Efficient and Accurate Explanation Estimation with Distribution Compression

TL;DR

The paper tackles the challenge of efficiently and accurately estimating post-hoc explanations at scale by showing that standard i.i.d. sampling introduces non-negligible approximation error. It introduces Compress Then Explain (CTE), which replaces background sampling with distribution compression via kernel thinning to form a small, representative coreset that closely matches the marginal data distribution. The authors prove bounds showing that explanation error is controlled by the maximum mean discrepancy (MMD) between the compressed and full distributions and demonstrate that CTE reduces sample complexity by 2–3x while improving stability across multiple explanation methods (e.g., SHAP, SAGE, Expected Gradients) on diverse datasets. The approach is a lightweight plug-in with negligible overhead that can be applied to a broad class of explanations, enabling more reliable and scalable explainability in real-world ML systems.

Abstract

We discover a theoretical connection between explanation estimation and distribution compression that significantly improves the approximation of feature attributions, importance, and effects. While the exact computation of various machine learning explanations requires numerous model inferences and becomes impractical, the computational cost of approximation increases with an ever-increasing size of data and model parameters. We show that the standard i.i.d. sampling used in a broad spectrum of algorithms for post-hoc explanation leads to an approximation error worthy of improvement. To this end, we introduce Compress Then Explain (CTE), a new paradigm of sample-efficient explainability. It relies on distribution compression through kernel thinning to obtain a data sample that best approximates its marginal distribution. CTE significantly improves the accuracy and stability of explanation estimation with negligible computational overhead. It often achieves an on-par explanation approximation error 2-3x faster by using fewer samples, i.e. requiring 2-3x fewer model evaluations. CTE is a simple, yet powerful, plug-in for any explanation method that now relies on i.i.d. sampling.
Paper Structure (5 sections, 2 theorems, 2 equations, 1 figure)

This paper contains 5 sections, 2 theorems, 2 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Sampling from the dataset is prevalent in explanation estimation
  4. Background on distribution compression
  5. Compress Then Explain (CTE)

Key Result

Proposition 1

For two empirical distributions $q_{\mathbb{X}}, q_{\widetilde{\mathbb{X}}}$ approximated with a kernel density estimator $\mathbf{k}$, we have $|f(\mathbf{x}_s; q_{\mathbb{X}}) - f(\mathbf{x}_s; q_{\widetilde{\mathbb{X}}})| \leq C_f \cdot \widehat{\mathrm{MMD}}_{\mathbf{k}}(q_{\mathbb{X}}, q_{\wide

Figures (1)

  • Figure 1: Garbage sample in, garbage explanation out.Sample then explain is a conventional approach to decrease the computational cost of explanation estimation. Although fast, sampling is inefficient and prone to error, which may even lead to changes in feature importance rankings. We propose compress then explain (cte), a new paradigm for accurate, yet efficient, estimation of explanations based on a marginal distribution that is compressed, e.g. with kernel thinning.

Theorems & Definitions (7)

  • Definition 1: Feature marginalization
  • Definition 2: Kernel maximum mean discrepancy gretton2012kerneldwivedi2021kernel
  • Definition 3: Biased estimator of $\mathrm{MMD}\xspace_{\mathbf{k}}$
  • Definition 4: Local explanation based on feature marginalization
  • Proposition 1: Feature marginalization is bounded by the maximum mean discrepancy between data samples
  • Definition 5: Global explanation
  • Proposition 2: Global explanation is bounded by the maximum mean discrepancy between data samples