Model-free generalized fiducial inference

TL;DR

This work builds a formal bridge between Conformal Prediction and generalized fiducial inference through imprecise probabilities, introducing a model-free GF framework that yields CP-compatible prediction sets and posterior-like lower/upper probabilities without relying on a fixed data model. By leveraging a rank-based, exchangeable, nonconformity-driven construction, the approach produces a GF contour whose level sets reproduce CP sets and simultaneously encodes epistemic uncertainty via a credal set. The paper connects CP-GF to NPI, conformal predictive distributions, and IMs, showing both theoretical guarantees (type 1 validity and concentration results) and practical tools such as pignistic transformations based on maximum entropy. The results provide a versatile, calibration-aware lens for uncertainty quantification with finite-sample guarantees, and propose computational schemes (e.g., MED-based posteriors) to obtain precise inferences from imprecise GF foundations. The work thus advances a coherent framework for model-free uncertainty quantification with clear avenues for future extensions in conditional validity and decision-theoretic mappings.

Abstract

Conformal prediction (CP) was developed to provide finite-sample probabilistic prediction guarantees. While CP algorithms are a relatively general-purpose approach to uncertainty quantification, with finite-sample guarantees, they lack versatility. Namely, the CP approach does not {\em prescribe} how to quantify the degree to which a data set provides evidence in support of (or against) an arbitrary event from a general class of events. In this paper, tools are offered from imprecise probability theory to build a formal connection between CP and generalized fiducial (GF) inference. These new insights establish a more general inferential lens from which CP can be understood, and demonstrate the pragmatism of fiducial ideas. The formal connection establishes a context in which epistemically-derived GF probability matches aleatoric/frequentist probability. Beyond this fact, it is illustrated how tools from imprecise probability theory, namely lower and upper probability functions, can be applied in the context of the imprecise GF distribution to provide posterior-like, prescriptive inference that is not possible within the CP framework alone. In addition to the primary CP generalization that is contributed, fundamental connections are synthesized between this new model-free GF and three other areas of contemporary research: nonparametric predictive inference (NPI), conformal predictive systems/distributions, and inferential models (IMs).

Figures (5)

• Figure 1: Hypothetical observed univariate data with $y_{1} = 4$, $y_{2} = 5$, and $n = 2$. With nonconformity measure $\Psi(y^{n+1}_{-i},y_{i}) := |\text{mean}(y_{-i}^{n+1}) - y_{i}|$, the inner black region represents the values of $y_{n+1}$ that would have rank 1 (i.e., $A_{n}(1) = \{ y \, : \, \text{rank}[t_{n+1}(y)] = 1 \}$), the outer grey region represents the values of $y_{n+1} \in A_{n}(2) = \{ y \, : \, \text{rank}[t_{n+1}(y)] = 2 \}$, and the outermost white region represents the values of $y_{n+1} \in A_{n}(n+1) = \{ y \, : \, \text{rank}[t_{n+1}(y)] = n+1 \}$.
• Figure 2: Both panels display plots of the GF contour, $f_{n}(y) = \mu\{ \Omega_{n}(V^{\star}) \ni y \}$; the left and right plots are based on samples of $n = 100$ realizations from the standard Gaussian and standard Cauchy distributions, respectively.
• Figure 3: Histograms of samples from $\pi_{y}^{n}$ computed by Algorithm \ref{['eq:mfgf_alg']}, and based on data sets of size $n = 10,000$ from the standard Gaussian distribution (left panel), standard Cauchy distribution (middle panel), and a mixture distribution (right panel). The nonconformity measure is $t(y_{i}) := y_{i}$. The black lines are plots of the respective density functions associated with the data.
• Figure 4: Based on the $n = 100$ data points drawn from the standard Gaussian distribution, as summarized by the histogram in the left panel, the middle and right panels display histograms of samples of size 10,000 drawn, respectively, from Algorithm \ref{['eq:mfgf_alg']} and its analogue by replacing $A_{n}(v^{\star})$ with $\Omega_{n}(v^{\star})$. The nonconformity measure is $t_{i}(y_{i}) := |\text{mean}(y_{-i}^{n+1}) - y_{i}|$. For reference, the contour function is provided as the black line in the middle and right panels.
• Figure 5: Based on the $n = 100$ data points drawn from a mixture of two Gaussian distributions, as summarized by the histogram in the left panel, the middle and right panels display histograms of samples of size 10,000 drawn, respectively, from Algorithm \ref{['eq:mfgf_alg']} and its analogue by replacing $A_{n}(v^{\star})$ with $\Omega_{n}(v^{\star})$. The nonconformity measure is $t_{i}(y_{i}) := |\text{mean}(y_{-i}^{n+1}) - y_{i}|$. For reference, the contour function is provided as the black line in the middle and right panels.

Theorems & Definitions (14)

• Definition 1: Type 1 validity -- cella2022
• Definition 2: Exchangeability
• Theorem 3: vovk2005
• Definition 4
• Definition 5: hannig2016
• Remark 6
• Theorem 7
• Corollary 8
• Lemma 9
• Theorem 10