Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Prior- and likelihood-free probabilistic inference with finite-sample calibration guarantees

Leonardo Cella, Emily C. Hector

Abstract

Motivated by parametric models for which the likelihood is analytically unavailable, numerically unstable, or prohibitively expensive to compute or optimize, we develop a prior- and likelihood-free framework for fully probabilistic (Bayesian-like) uncertainty quantification with finite-sample calibration guarantees. Our method, a type of inferential model, produces data-dependent degrees of belief about claims concerning the unknown parameter while controlling the frequency with which high belief is assigned to false claims, even in finite-sample settings. Our procedure is general in that it requires only the ability to simulate from the model. We first rank candidate parameter values according to how well data simulated from the model agree with the observed data, and then rescale these rankings in a way that yields the desired finite-sample calibration guarantees. The key idea is to employ a permutation-invariant function, such as a depth function, to rank parameter values. We show that such a choice yields closed-form calibration rescaling calculations, making the procedure computationally simple. We illustrate our method's broad appeal with four examples, including differential privacy and Ising models. An analysis of the spatial configuration of 2025 measles outbreaks in the U.S. showcases our method's practical advantages.

Prior- and likelihood-free probabilistic inference with finite-sample calibration guarantees

Abstract

Motivated by parametric models for which the likelihood is analytically unavailable, numerically unstable, or prohibitively expensive to compute or optimize, we develop a prior- and likelihood-free framework for fully probabilistic (Bayesian-like) uncertainty quantification with finite-sample calibration guarantees. Our method, a type of inferential model, produces data-dependent degrees of belief about claims concerning the unknown parameter while controlling the frequency with which high belief is assigned to false claims, even in finite-sample settings. Our procedure is general in that it requires only the ability to simulate from the model. We first rank candidate parameter values according to how well data simulated from the model agree with the observed data, and then rescale these rankings in a way that yields the desired finite-sample calibration guarantees. The key idea is to employ a permutation-invariant function, such as a depth function, to rank parameter values. We show that such a choice yields closed-form calibration rescaling calculations, making the procedure computationally simple. We illustrate our method's broad appeal with four examples, including differential privacy and Ising models. An analysis of the spatial configuration of 2025 measles outbreaks in the U.S. showcases our method's practical advantages.
Paper Structure (16 sections, 5 theorems, 24 equations, 6 figures, 1 table)

This paper contains 16 sections, 5 theorems, 24 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Likelihood-based IMs: construction, limitations, and desiderata
  3. Problem setup and motivation
  4. Key insights towards likelihood-free IMs
  5. Likelihood-free IMs
  6. Construction
  7. Ranking step
  8. Validification step
  9. Finite-sample guarantees
  10. Synthetic data examples
  11. Correlation coefficient
  12. $g$-and-$k$ distributions
  13. Differential privacy
  14. Ising model
  15. Real data example
  16. ...and 1 more sections

Key Result

Corollary 1

CellaMartinMainSeverity For an IM whose contour satisfies the validity property in eq:IMvalidity, the following is true for all $\alpha \in [0,1]$:

Figures (6)

  • Figure 1: Results shown in black correspond to the likelihood-based contour. Results shown in blue, red, and green correspond to the likelihood-free contours based on the sample correlation coefficient, the two-dimensional sufficient statistic, and a single sufficient statistic, respectively.
  • Figure 2: Panel (a): Histogram of a dataset, n=100. Panel (b): In blue, the 0.2-, 0.5-, and 0.8-level sets of the likelihood-free IM contour. In black, the 20%, 50%, and 80% highest posterior density regions of the ABC solution. Solid, dashed, and dotted lines correspond to increasing levels. Panel (c): Distribution functions of the IM lower probabilities (dashed) and Bayes posterior probabilities (solid) assigned to $C_1$ (blue), $C_2$ (green) and $C_3$ (red), based on 500 simulation replicates.
  • Figure 3: Likelihood-based contour (black) and likelihood-free contour (blue).
  • Figure 4: Panel (a): Observed data from a $4\times 4$ Ising model, where light gray cells represent $-1$ and dark gray cells represent $+1$. Panel (b): Likelihood-based contour (black) and likelihood-free contour (blue) based on data in (a).
  • Figure 5: Panel (a): Observed data from a $20\times 20$ Ising model, where light gray cells represent $-1$ and dark gray cells represent $+1$. Panel (b): 0.1-level sets of the likelihood-free IM contour using Tukey (blue) and Mahalanobis (red) depths. Panel (c): Empirical distribution function of the likelihood-free IMs contours evaluated at the true $\Theta$, using Tukey (blue) and Mahalanobis (red) depths, based on 500 simulation replicates.
  • ...and 1 more figures

Theorems & Definitions (9)

  • Corollary 1
  • Lemma 1
  • proof
  • Theorem 1
  • proof
  • Corollary 2
  • proof
  • Corollary 3
  • proof