Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SurrogateSHAP: Training-Free Contributor Attribution for Text-to-Image (T2I) Models

Mingyu Lu, Soham Gadgil, Chris Lin, Chanwoo Kim, Su-In Lee

TL;DR

SurrogateSHAP tackles the challenge of attributing value to data contributors in Text-to-Image diffusion models using the Shapley value. It replaces expensive retraining with a training-free proxy game that constructs subset outputs as mixtures of conditionals, and it learns a gradient-boosted tree surrogate to estimate Shapley values via TreeSHAP. The method delivers high fidelity to retraining utilities while dramatically reducing computation, validated on CIFAR-20, ArtBench, Fashion-Product, and a dermatology data auditing case study. By enabling scalable, principled contributor valuation, SurrogateSHAP facilitates fair data marketplaces and safety auditing, while also highlighting governance considerations around data pruning based on attribution.

Abstract

As Text-to-Image (T2I) diffusion models are increasingly used in real-world creative workflows, a principled framework for valuing contributors who provide a collection of data is essential for fair compensation and sustainable data marketplaces. While the Shapley value offers a theoretically grounded approach to attribution, it faces a dual computational bottleneck: (i) the prohibitive cost of exhaustive model retraining for each sampled subset of players (i.e., data contributors) and (ii) the combinatorial number of subsets needed to estimate marginal contributions due to contributor interactions. To this end, we propose SurrogateSHAP, a retraining-free framework that approximates the expensive retraining game through inference from a pretrained model. To further improve efficiency, we employ a gradient-boosted tree to approximate the utility function and derive Shapley values analytically from the tree-based model. We evaluate SurrogateSHAP across three diverse attribution tasks: (i) image quality for DDPM-CFG on CIFAR-20, (ii) aesthetics for Stable Diffusion on Post-Impressionist artworks, and (iii) product diversity for FLUX.1 on Fashion-Product data. Across settings, SurrogateSHAP outperforms prior methods while substantially reducing computational overhead, consistently identifying influential contributors across multiple utility metrics. Finally, we demonstrate that SurrogateSHAP effectively localizes data sources responsible for spurious correlations in clinical images, providing a scalable path toward auditing safety-critical generative models.

SurrogateSHAP: Training-Free Contributor Attribution for Text-to-Image (T2I) Models

TL;DR

SurrogateSHAP tackles the challenge of attributing value to data contributors in Text-to-Image diffusion models using the Shapley value. It replaces expensive retraining with a training-free proxy game that constructs subset outputs as mixtures of conditionals, and it learns a gradient-boosted tree surrogate to estimate Shapley values via TreeSHAP. The method delivers high fidelity to retraining utilities while dramatically reducing computation, validated on CIFAR-20, ArtBench, Fashion-Product, and a dermatology data auditing case study. By enabling scalable, principled contributor valuation, SurrogateSHAP facilitates fair data marketplaces and safety auditing, while also highlighting governance considerations around data pruning based on attribution.

Abstract

As Text-to-Image (T2I) diffusion models are increasingly used in real-world creative workflows, a principled framework for valuing contributors who provide a collection of data is essential for fair compensation and sustainable data marketplaces. While the Shapley value offers a theoretically grounded approach to attribution, it faces a dual computational bottleneck: (i) the prohibitive cost of exhaustive model retraining for each sampled subset of players (i.e., data contributors) and (ii) the combinatorial number of subsets needed to estimate marginal contributions due to contributor interactions. To this end, we propose SurrogateSHAP, a retraining-free framework that approximates the expensive retraining game through inference from a pretrained model. To further improve efficiency, we employ a gradient-boosted tree to approximate the utility function and derive Shapley values analytically from the tree-based model. We evaluate SurrogateSHAP across three diverse attribution tasks: (i) image quality for DDPM-CFG on CIFAR-20, (ii) aesthetics for Stable Diffusion on Post-Impressionist artworks, and (iii) product diversity for FLUX.1 on Fashion-Product data. Across settings, SurrogateSHAP outperforms prior methods while substantially reducing computational overhead, consistently identifying influential contributors across multiple utility metrics. Finally, we demonstrate that SurrogateSHAP effectively localizes data sources responsible for spurious correlations in clinical images, providing a scalable path toward auditing safety-critical generative models.
Paper Structure (65 sections, 1 theorem, 38 equations, 30 figures, 15 tables, 1 algorithm)

This paper contains 65 sections, 1 theorem, 38 equations, 30 figures, 15 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminaries and Problem Setup
  4. Text-to-Image (T2I) Diffusion Models
  5. Attribute Contributor via Shapley value
  6. Computational bottlenecks
  7. Method - SurrogateSHAP
  8. Training-Free Coalition Evaluation
  9. Training-Free Proxy Game
  10. Theoretical Fidelity of the Proxy
  11. Approximate Shapley Values via Tree Explainer
  12. Experiments Setup & Results
  13. Datasets & Models
  14. Proxy Game Fidelity
  15. Sample Efficiency on Synthetic Games
  16. ...and 50 more sections

Key Result

Proposition 4.2

Under Assumption asm:stability, recall that $v(S):=\mathcal{F}(p_{\theta^*_S})$ and $\hat{v}_{\theta}(S):=\mathcal{F}(p_{\theta})$, where the utility functional $\mathcal{F}$ is induced by $\mathcal{V}$ via and $\mathcal{V}$ is a utility functional on probability measures over $\mathbb{R}^{m_u}$. If $\mathcal{V}$ is $L_u$-Lipschitz with respect to the $W_p$ metric for some $p\in \{ 1,2\}$ on $\ma

Figures (30)

  • Figure 1: Alignment between the proxy utility $\hat{v}_\theta(S)$ (y-axis) and retraining utility $v(S)$ (x-axis) across subsets, for (a) FID, (b) Inception Score (IS), and (c) mean aesthetic score.
  • Figure 2: (a) Local samples and structural similarities between retrained models (left) and the target model (right) for three subsets. (b) Shapley estimator accuracy: $\ell_2$ error to the oracle versus subset budget on synthetic games (mean $\pm$ std over 60 independent trials).
  • Figure 3: LDS (%) at $\alpha=0.5$ for Shapley-based methods, evaluated on (a) FID and (b) average aesthetic score. Computational budget is measured in units of retraining-and-inference runtime.
  • Figure 4: (a) SurrogateSHAP value for ten hospitals (x-axis: magnitude; y-axis: site; red/blue: positive/negative). (b) Examples of dermoscopic images with visual artifacts.
  • Figure S.1: Sample-efficiency across utility landscapes. The x-axis shows the number of sampled coalitions $M$, and the y-axis shows the $\ell_2$ relative to the oracle. (a) Simple linear additive utilities; (b) nonlinear utilities; (c) interaction-dominated utilities.
  • ...and 25 more figures

Theorems & Definitions (4)

  • Definition 3.1: Contributor Attribution
  • Proposition 4.2: Proxy Error Bound
  • Remark 4.3: Metric Applicability
  • proof : Proof of Proposition \ref{['prop:proxy-error-bound']}