Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ExplainerPFN: Towards tabular foundation models for model-free zero-shot feature importance estimations

Joao Fonseca, Julia Stoyanovich

TL;DR

This work tackles the problem of explaining predictions without accessing the underlying model by proposing ExplainerPFN, a tabular foundation model that learns to predict Shapley-style feature attributions from input-label pairs alone. Trained on synthetic SCM-driven data, ExplainerPFN operates in a fully zero-shot regime, delivering per-instance attributions $\phi_i^j$ conditioned on $(X,\hat{Y})$ without querying $f$ or using reference SHAP explanations. The method achieves competitive fidelity to SHAP in both zero-shot and few-shot regimes, while offering substantial speed advantages over traditional SHAP computations, especially for large or inaccessible models. Through extensive experiments on real and synthetic tabular datasets, the paper demonstrates the practicality and potential of data-distribution-driven attribution patterns for scalable, model-free explanations with broad implications for auditing and transparency in constrained settings.

Abstract

Computing the importance of features in supervised classification tasks is critical for model interpretability. Shapley values are a widely used approach for explaining model predictions, but require direct access to the underlying model, an assumption frequently violated in real-world deployments. Further, even when model access is possible, their exact computation may be prohibitively expensive. We investigate whether meaningful Shapley value estimations can be obtained in a zero-shot setting, using only the input data distribution and no evaluations of the target model. To this end, we introduce ExplainerPFN, a tabular foundation model built on TabPFN that is pretrained on synthetic datasets generated from random structural causal models and supervised using exact or near-exact Shapley values. Once trained, ExplainerPFN predicts feature attributions for unseen tabular datasets without model access, gradients, or example explanations. Our contributions are fourfold: (1) we show that few-shot learning-based explanations can achieve high fidelity to SHAP values with as few as two reference observations; (2) we propose ExplainerPFN, the first zero-shot method for estimating Shapley values without access to the underlying model or reference explanations; (3) we provide an open-source implementation of ExplainerPFN, including the full training pipeline and synthetic data generator; and (4) through extensive experiments on real and synthetic datasets, we show that ExplainerPFN achieves performance competitive with few-shot surrogate explainers that rely on 2-10 SHAP examples.

ExplainerPFN: Towards tabular foundation models for model-free zero-shot feature importance estimations

TL;DR

This work tackles the problem of explaining predictions without accessing the underlying model by proposing ExplainerPFN, a tabular foundation model that learns to predict Shapley-style feature attributions from input-label pairs alone. Trained on synthetic SCM-driven data, ExplainerPFN operates in a fully zero-shot regime, delivering per-instance attributions conditioned on without querying or using reference SHAP explanations. The method achieves competitive fidelity to SHAP in both zero-shot and few-shot regimes, while offering substantial speed advantages over traditional SHAP computations, especially for large or inaccessible models. Through extensive experiments on real and synthetic tabular datasets, the paper demonstrates the practicality and potential of data-distribution-driven attribution patterns for scalable, model-free explanations with broad implications for auditing and transparency in constrained settings.

Abstract

Computing the importance of features in supervised classification tasks is critical for model interpretability. Shapley values are a widely used approach for explaining model predictions, but require direct access to the underlying model, an assumption frequently violated in real-world deployments. Further, even when model access is possible, their exact computation may be prohibitively expensive. We investigate whether meaningful Shapley value estimations can be obtained in a zero-shot setting, using only the input data distribution and no evaluations of the target model. To this end, we introduce ExplainerPFN, a tabular foundation model built on TabPFN that is pretrained on synthetic datasets generated from random structural causal models and supervised using exact or near-exact Shapley values. Once trained, ExplainerPFN predicts feature attributions for unseen tabular datasets without model access, gradients, or example explanations. Our contributions are fourfold: (1) we show that few-shot learning-based explanations can achieve high fidelity to SHAP values with as few as two reference observations; (2) we propose ExplainerPFN, the first zero-shot method for estimating Shapley values without access to the underlying model or reference explanations; (3) we provide an open-source implementation of ExplainerPFN, including the full training pipeline and synthetic data generator; and (4) through extensive experiments on real and synthetic datasets, we show that ExplainerPFN achieves performance competitive with few-shot surrogate explainers that rely on 2-10 SHAP examples.
Paper Structure (28 sections, 13 equations, 7 figures, 6 tables)

This paper contains 28 sections, 13 equations, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Preliminaries and Background
  3. Problem Formulation
  4. ExplainerPFN
  5. Tabular Transformer Encoder
  6. Generating Training Data
  7. Training Objective
  8. Improving Attribution Consistency via Shapley Properties
  9. Addressing Explanation Multiplicity
  10. Experiments
  11. Few-shot Feature Importance Estimation
  12. Zero-shot Feature Importance Estimation
  13. Computational Efficiency
  14. Case study: ACS Public Coverage
  15. Limitations and Future Work
  16. ...and 13 more sections

Figures (7)

  • Figure 1: Comparison between ExplainerPFN and SHAP feature-importance attributions across the worst and best performing cases for ExplainerPFN, using the RF as base model: (a) Abalone (AB) and (b) Banknote Authentication (BA).
  • Figure 2: Pearson correlation analysis between ExplainerPFN and SHAP feature importance attribution on synthetic datasets, as a function of the number of features using different correction steps. On average, no post-processing (blue) outperforms instance-level efficiency correction (orange), but both are consistently outperformed by applying all correction steps (green; mean-zero attributions, variance decomposition, and instance-level efficiency).
  • Figure 3: Per-feature comparison between ExplainerPFN and SHAP's feature importance attributions for predicting health insurance coverage using the ACS Public Coverage dataset.
  • Figure 4: Overall comparison between ExplainerPFN and SHAP's feature importance attributions for predicting health insurance coverage in the ACS Public Coverage dataset.
  • Figure 5: DAG reconstruction analyses. (a) Example of a ground truth DAG. The nodes/features marked in red are removed after generating synthetic data, but before feeding the resulting data to TabPFN. (b) Estimated DAG using ExplainerPFN with the highest-scoring 7 edges, based on feature contributions estimated by ExplainerPFN. (c) Graph Edit Distance between the true and reconstructed DAGs as a function of the number of edges kept when reconstructing the DAG using ExplainerPFN over 50 synthetic datasets generated with randomly sampled DAGs.
  • ...and 2 more figures