Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Non-parametric finite-sample credible intervals with one-dimensional priors: a middle ground between Bayesian and frequentist intervals

Tim Ritmeester

TL;DR

This paper introduces non-parametric finite-sample credible intervals that require only one-dimensional priors, providing a practical middle ground between Bayesian and frequentist intervals. It defines a data-to-belief algorithm that ensures validity after interval reporting and demonstrates two concrete non-parametric instantiations for CDF estimation and mean estimation on bounded support, with explicit constructions based on a weighted Bayesian integral $A = \frac{\int l(\theta) b(\theta)\,d\theta}{\int l(\theta) b(\theta)\,d\theta}$. The CDF case achieves equality in the validity condition and asymptotically matches the Clopper–Pearson width, while the mean case uses a carefully chosen noise term to bound the interval and is asymptotically wider than Hoeffding-based frequentist bounds by quantified amounts. The work analyzes asymptotic behavior, compares with standard methods, discusses practical advantages (such as prior-driven narrowness in small samples and natural sequential analysis), and outlines future directions to expand the approach to additional non-parametric problems and fiducial connections.

Abstract

We propose a new type of statistical interval obtained by weakening the definition of a p% credible interval: Having observed the interval (rather than the full dataset) we should put at least a p% belief in it. From a decision-theoretical point of view the resulting intervals occupy a middle ground between frequentist and fully Bayesian statistical intervals, both practically and philosophically: To a p% Bayesian credible interval we should assign (at least a) p% belief also after seeing the full dataset, while p% frequentist intervals we in general only assign a p% belief before seeing either the data or the interval. We derive concrete implementations for two cases: estimation of the fraction of a distribution that falls below a certain value (i.e., the CDF), and of the mean of a distribution with bounded support. Even though the problems are fully non parametric, these methods require only one-dimensional priors. They share many of the practical advantages of Bayesian methods while avoiding the complexity of assigning high-dimensional priors altogether. Asymptotically they give intervals equivalent to the fully Bayesian approach and somewhat wider intervals, respectively. We discuss promising directions where the proposed type of interval may provide significant advantages.

Non-parametric finite-sample credible intervals with one-dimensional priors: a middle ground between Bayesian and frequentist intervals

TL;DR

This paper introduces non-parametric finite-sample credible intervals that require only one-dimensional priors, providing a practical middle ground between Bayesian and frequentist intervals. It defines a data-to-belief algorithm that ensures validity after interval reporting and demonstrates two concrete non-parametric instantiations for CDF estimation and mean estimation on bounded support, with explicit constructions based on a weighted Bayesian integral . The CDF case achieves equality in the validity condition and asymptotically matches the Clopper–Pearson width, while the mean case uses a carefully chosen noise term to bound the interval and is asymptotically wider than Hoeffding-based frequentist bounds by quantified amounts. The work analyzes asymptotic behavior, compares with standard methods, discusses practical advantages (such as prior-driven narrowness in small samples and natural sequential analysis), and outlines future directions to expand the approach to additional non-parametric problems and fiducial connections.

Abstract

We propose a new type of statistical interval obtained by weakening the definition of a p% credible interval: Having observed the interval (rather than the full dataset) we should put at least a p% belief in it. From a decision-theoretical point of view the resulting intervals occupy a middle ground between frequentist and fully Bayesian statistical intervals, both practically and philosophically: To a p% Bayesian credible interval we should assign (at least a) p% belief also after seeing the full dataset, while p% frequentist intervals we in general only assign a p% belief before seeing either the data or the interval. We derive concrete implementations for two cases: estimation of the fraction of a distribution that falls below a certain value (i.e., the CDF), and of the mean of a distribution with bounded support. Even though the problems are fully non parametric, these methods require only one-dimensional priors. They share many of the practical advantages of Bayesian methods while avoiding the complexity of assigning high-dimensional priors altogether. Asymptotically they give intervals equivalent to the fully Bayesian approach and somewhat wider intervals, respectively. We discuss promising directions where the proposed type of interval may provide significant advantages.
Paper Structure (10 sections, 17 equations, 2 figures, 2 tables)

This paper contains 10 sections, 17 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. The general approach
  3. The proposed intervals and their properties
  4. derivation
  5. Asymptotic analysis for estimation of the CDF
  6. Asymptotic analysis for estimation of the mean
  7. Comparison to standard Bayesian and frequentist methods
  8. Discussion and next steps
  9. Numerical verification
  10. Additional analytical details for estimation of the mean.

Figures (2)

  • Figure 1: Validity: The belief value $p$ assigned to the interval by the algorithm, and the resulting belief $b(\theta_I | A = p)$ that the user has in this interval after seeing this result, calculated numerically for a specific prior over distributions (as specified in App. \ref{['sec: numerical_verification']}). In Sec. \ref{['sec: derivation']} we derived this to be a valid credible interval in the sense of Eq. \ref{['eq: validity']}, i.e., that the latter is at least equal to the former. To calculate $b(\theta_I | A = p)$ we used an $ABC$ rejection sampler mikael_2013 with $\epsilon = 0.02$ and the number of samples equal to $2000$. See App \ref{['sec: numerical_verification']} for further details.
  • Figure 2: Precision: The width of the proposed credible intervals as a function of the number of samples $N$, compared to the width of standard frequentist intervals for the same estimation (Clopper-Pearson (a) clopper_1934 and Hoeffding (b) wasserman2006all). The frequentist intervals can in turn asymptotically be related to the width of the fully Bayesian estimate; see Sec. \ref{['sec: proposed_intervals']}. For small $N$ the proposed intervals are narrower, while asymptotically they become (a) equally large or (b) somewhat wider, matching the asymptotic results presented in Sec. \ref{['sec: proposed_intervals']}. See App \ref{['sec: numerical_verification']} for further details on the numerical simulations.