Shapley Value on Uncertain Data

TL;DR

This work generalizes data Shapley valuation to settings where each participant contributes data drawn from a distribution, making the Shapley value itself a random variable. It formalizes the probabilistic Shapley value and derives unbiased estimators for its mean $\mathbb{E}[\phi_i]$ and variance $\mathrm{Var}(\phi_i)$, along with closed-form results under tractable marginal contributions. Three scalable Monte Carlo methods are introduced: a baseline independent-sampling estimator, a pooled estimator that reuses fixed data pools, and a stratified pooled estimator that allocates sampling budget adaptively to high-variance players. Empirical results on synthetic and real data show strong accuracy-efficiency trade-offs, with the stratified pooled method delivering substantial variance reductions at low additional cost. The framework enables robust, uncertainty-aware data valuation for stochastic data-sharing environments such as data markets and privacy-preserving or federated systems.

Abstract

The Shapley value provides a principled framework for fairly distributing rewards among participants according to their individual contributions. While prior work has applied this concept to data valuation in machine learning, existing formulations overwhelmingly assume that each participant contributes a fixed, deterministic dataset. In practice, however, data owners often provide samples drawn from underlying probabilistic distributions, introducing stochasticity into their marginal contributions and rendering the Shapley value itself a random variable. This work addresses this gap by proposing a framework for the Shapley value of probabilistic data distributions that quantifies both the expected contribution and the variance of each participant, thereby capturing uncertainty induced by random sampling. We develop theoretical and empirical methodologies for estimating these quantities: on the theoretical side, we derive unbiased estimators for the expectation and variance of the probabilistic Shapley value and analyze their statistical properties; on the empirical side, we introduce three Monte Carlo-based estimation algorithms - a baseline estimator using independent samples, a pooled estimator that improves efficiency through sample reuse, and a stratified pooled estimator that adaptively allocates sampling budget based on player-specific variability. Experiments on synthetic and real datasets demonstrate that these methods achieve strong accuracy-efficiency trade-offs, with the stratified pooled approach attaining substantial variance reduction at minimal additional cost. By extending Shapley value analysis from deterministic datasets to probabilistic data distributions, this work provides both theoretical rigor and practical tools for fair and reliable data valuation in modern stochastic data-sharing environments.

Figures (4)

• Figure 1: Effect of $n_{\text{games}}$ and $n_{\text{iter}}$ on variance metrics for the baseline method. Increasing either parameter reduces both variance metrics across datasets, with $n_{\text{games}}$ dominating mean stability and $n_{\text{iter}}$ dominating variance stability.
• Figure 2: Effect of pool size on pooled method variance for the Wine Quality dataset ($n_{\text{boot}} = 60$). Larger pools reduce both variance metrics by 40--86%, with stronger improvements at higher $n_{\text{iter}}$.
• Figure 3: Comparison of baseline and pooled methods. Variance decreases consistently with larger $n_{\text{games}}$ and $n_{\text{iter}}$. The pooled method achieves comparable or lower variance with over 97% fewer samples.
• Figure 4: Variance comparison of stratified pooled method with baseline and pooled methods on Wine Quality data. The stratified method consistently achieves lower variance across budgets, with larger gains at higher $n_{\text{iter}}$.

Theorems & Definitions (13)

• Definition 2.1: Shapley value shapley1953value
• Proposition 3.1
• proof
• Example 3.1
• Proposition 3.2
• proof
• Example 3.2
• Proposition 3.3
• proof
• Proposition 3.4