Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Computing Conditional Shapley Values Using Tabular Foundation Models

Lars Henry Berge Olsen, Dennis Christensen

TL;DR

This work tackles the computational bottleneck of computing conditional Shapley values in explainable AI by leveraging TabPFN, a tabular foundation model that uses in‑context learning to approximate conditional expectations without retraining. It evaluates two regression paradigms—separate regression and surrogate regression—within both TabPFN variants and traditional baselines on simulated Gaussian data and five real datasets, using metrics like MAE and $\operatorname{MSE}_v$ to assess accuracy and runtime. The results show that TabPFN often yields the best or near‑best accuracy, especially for smooth predictive functions, while delivering substantial speedups; surrogate approaches are generally weaker due to missingness patterns not captured in pretraining. The findings underscore the potential of tabular foundation models for scalable, model‑agnostic Shapley value estimation and point to future directions such as pretraining on data with realistic missingness patterns and expanding context sizes. Overall, the paper demonstrates that TabPFN is a competitive and efficient tool for conditional Shapley value estimation in tabular settings, with clear guidance on when to favor separate versus surrogate regression configurations.

Abstract

Shapley values have become a cornerstone of explainable AI, but they are computationally expensive to use, especially when features are dependent. Evaluating them requires approximating a large number of conditional expectations, either via Monte Carlo integration or regression. Until recently it has not been possible to fully exploit deep learning for the regression approach, because retraining for each conditional expectation takes too long. Tabular foundation models such as TabPFN overcome this computational hurdle by leveraging in-context learning, so each conditional expectation can be approximated without any re-training. In this paper, we compute Shapley values with multiple variants of TabPFN and compare their performance with state-of-the-art methods on both simulated and real datasets. In most cases, TabPFN yields the best performance; where it does not, it is only marginally worse than the best method, at a fraction of the runtime. We discuss further improvements and how tabular foundation models can be better adapted specifically for conditional Shapley value estimation.

Computing Conditional Shapley Values Using Tabular Foundation Models

TL;DR

This work tackles the computational bottleneck of computing conditional Shapley values in explainable AI by leveraging TabPFN, a tabular foundation model that uses in‑context learning to approximate conditional expectations without retraining. It evaluates two regression paradigms—separate regression and surrogate regression—within both TabPFN variants and traditional baselines on simulated Gaussian data and five real datasets, using metrics like MAE and to assess accuracy and runtime. The results show that TabPFN often yields the best or near‑best accuracy, especially for smooth predictive functions, while delivering substantial speedups; surrogate approaches are generally weaker due to missingness patterns not captured in pretraining. The findings underscore the potential of tabular foundation models for scalable, model‑agnostic Shapley value estimation and point to future directions such as pretraining on data with realistic missingness patterns and expanding context sizes. Overall, the paper demonstrates that TabPFN is a competitive and efficient tool for conditional Shapley value estimation in tabular settings, with clear guidance on when to favor separate versus surrogate regression configurations.

Abstract

Shapley values have become a cornerstone of explainable AI, but they are computationally expensive to use, especially when features are dependent. Evaluating them requires approximating a large number of conditional expectations, either via Monte Carlo integration or regression. Until recently it has not been possible to fully exploit deep learning for the regression approach, because retraining for each conditional expectation takes too long. Tabular foundation models such as TabPFN overcome this computational hurdle by leveraging in-context learning, so each conditional expectation can be approximated without any re-training. In this paper, we compute Shapley values with multiple variants of TabPFN and compare their performance with state-of-the-art methods on both simulated and real datasets. In most cases, TabPFN yields the best performance; where it does not, it is only marginally worse than the best method, at a fraction of the runtime. We discuss further improvements and how tabular foundation models can be better adapted specifically for conditional Shapley value estimation.
Paper Structure (10 sections, 9 equations, 1 figure, 2 tables)

This paper contains 10 sections, 9 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Shapley values
  3. Monte Carlo estimation
  4. Regression-Based Estimation
  5. Computing Shapley values with TabPFN
  6. Simulation Study
  7. Real data results
  8. Conclusion
  9. Acknowledgments.
  10. Disclosure of Interests.

Figures (1)

  • Figure 1: Results of the gam_ more_ interactions experiment: Boxplots of the mean absolute error between the true and estimated Shapley values for the test observations using different methods and for different dependence levels $\rho$. Abbreviations: GH = generalized hyperbolic, Ctree = conditional inference tree, VAEAC = variational autoencoders with arbitrary conditioning, LM = linear regression model, GAM = generalized additive model, PPR = projection pursuit regression, RF = random forest, NN = neural network. For a complete description of all models and appropriate references, see olsen2024comparative.