Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Uncertainty Quantification for Prior-Data Fitted Networks using Martingale Posteriors

Thomas Nagler, David Rügamer

TL;DR

This work addresses the lack of uncertainty quantification in Prior-Data Fitted Networks (PFNs) by introducing Martingale Posteriors (MPs) that construct Bayesian posteriors for PFN-derived predictive summaries. The authors extend the MP framework to conditional inference using a nonparametric Gaussian-copula update and flexible learning-rate schedules, providing convergence guarantees and a practical algorithm with $O(BN)$ complexity. Through extensive simulations and real-data benchmarks, they demonstrate that MP-based posteriors yield meaningful epistemic uncertainty, competitive coverage, and favorable runtimes compared to bootstrap, while highlighting biases in PFNs that constrain nominal coverage in some regimes. They also discuss limitations and potential extensions to other in-context learners, including LLMs, outlining a path for principled uncertainty quantification in large pretrained models beyond tabular PFNs.

Abstract

Prior-data fitted networks (PFNs) have emerged as promising foundation models for prediction from tabular data sets, achieving state-of-the-art performance on small to moderate data sizes without tuning. While PFNs are motivated by Bayesian ideas, they do not provide any uncertainty quantification for predictive means, quantiles, or similar quantities. We propose a principled and efficient sampling procedure to construct Bayesian posteriors for such estimates based on Martingale posteriors, and prove its convergence. Several simulated and real-world data examples showcase the uncertainty quantification of our method in inference applications.

Uncertainty Quantification for Prior-Data Fitted Networks using Martingale Posteriors

TL;DR

This work addresses the lack of uncertainty quantification in Prior-Data Fitted Networks (PFNs) by introducing Martingale Posteriors (MPs) that construct Bayesian posteriors for PFN-derived predictive summaries. The authors extend the MP framework to conditional inference using a nonparametric Gaussian-copula update and flexible learning-rate schedules, providing convergence guarantees and a practical algorithm with complexity. Through extensive simulations and real-data benchmarks, they demonstrate that MP-based posteriors yield meaningful epistemic uncertainty, competitive coverage, and favorable runtimes compared to bootstrap, while highlighting biases in PFNs that constrain nominal coverage in some regimes. They also discuss limitations and potential extensions to other in-context learners, including LLMs, outlining a path for principled uncertainty quantification in large pretrained models beyond tabular PFNs.

Abstract

Prior-data fitted networks (PFNs) have emerged as promising foundation models for prediction from tabular data sets, achieving state-of-the-art performance on small to moderate data sizes without tuning. While PFNs are motivated by Bayesian ideas, they do not provide any uncertainty quantification for predictive means, quantiles, or similar quantities. We propose a principled and efficient sampling procedure to construct Bayesian posteriors for such estimates based on Martingale posteriors, and prove its convergence. Several simulated and real-world data examples showcase the uncertainty quantification of our method in inference applications.
Paper Structure (45 sections, 3 theorems, 25 equations, 10 figures, 5 tables, 1 algorithm)

This paper contains 45 sections, 3 theorems, 25 equations, 10 figures, 5 tables, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. BACKGROUND
  3. Prior-Data Fitted Networks
  4. Bayesian Inference
  5. Martingale Posteriors
  6. EFFICIENT MARTINGALE POSTERIORS FOR PRIOR-DATA FITTED NETWORKS
  7. Martingale Posteriors for Conditional Inference
  8. Efficient PPD Updates Based on the Gaussian Copula
  9. Flexible Learning Rate Schedules
  10. Theoretical Properties
  11. The Role of $\beta$
  12. Computation
  13. NUMERICAL EXPERIMENTS
  14. TabPFN Martingale Property Check
  15. Results
  16. ...and 30 more sections

Key Result

Proposition 1

It holds $(y_{N + 1}, y_{N + 2}, \dots) \to_d (z_1, z_2, \dots)$ as $N \to \infty$ where $(z_1, z_2, \dots)$ has an exchangeable distribution.

Figures (10)

  • Figure 1: Visualization of the diffusion process data (left) together with the PFN predicted density (center) and the estimated epistemic uncertainty (right) via our martingale posterior approach.
  • Figure 2: PFN's initial CDF (x-axis) vs. its average CDF of subsequent iterations (y-axis).
  • Figure 3: Effect of $N$ on the convergence of $P_N$, depicted by $B=30$ different PPD mean estimates.
  • Figure 4: Mean values and standard deviations (error bars) of miscoverage (observed minus nominal value; x-axis) and interval length (y-axis) of the 90% credible intervals produced by the original method (blue), our proposed adaptions (orange, purple), and the bootstrap, across the different benchmark datasets. The red vertical line indicates perfect coverage.
  • Figure 5: Coverage and interval length behavior for the larger UCI datasets (rows) for differently sized training datasets.
  • ...and 5 more figures

Theorems & Definitions (3)

  • Proposition 1
  • Proposition 2
  • Proposition 3