Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Non-parametric Bayesian inference via loss functions under model misspecification

Yu Luo, David A. Stephens, Daniel J. Graham, Emma J. McCoy

TL;DR

This work develops a non-parametric Bayesian framework for inference when the data-generating process may be misspecified, by formulating inference as loss minimization and constructing Gibbs/posterior distributions via Dirichlet-process–based Bayesian non-parametric updating. It presents two computationally efficient, optimization-based updating schemes (prior-to-posterior and predictive-to-posterior) that converge to the same posterior under exchangeability and are shown to be consistent and asymptotically normal when misspecification is present. A calibration procedure for the learning rate ensures valid uncertainty quantification, and the authors apply the methods to causal inference via propensity-score regression, demonstrating doubly robust properties and robust performance in simulations and an empirical UK speed camera study. The results offer scalable, robust alternatives to traditional likelihood-based Bayesian updating in settings where model misspecification is a concern, with broad applicability to causal inference and predictive decision-making.

Abstract

In the usual Bayesian setting, a full probabilistic model is required to link the data and parameters, and the form of this model and the inference and prediction mechanisms are specified via de Finetti's representation. In general, such a formulation is not robust to model misspecification of its component parts. An alternative approach is to draw inference based on loss functions, where the quantity of interest is defined as a minimizer of some expected loss, and to construct posterior distributions based on the loss-based formulation; this strategy underpins the construction of the Gibbs posterior. We develop a Bayesian non-parametric approach; specifically, we generalize the Bayesian bootstrap, and specify a Dirichlet process model for the distribution of the observables. We implement this using direct prior-to-posterior calculations, but also using predictive sampling. We also study the assessment of posterior validity for non-standard Bayesian calculations. We show that the developed non-standard Bayesian updating procedures yield valid posterior distributions in terms of consistency and asymptotic normality under model misspecification. Simulation studies show that the proposed methods can recover the true value of the parameter under misspecification.

Non-parametric Bayesian inference via loss functions under model misspecification

TL;DR

This work develops a non-parametric Bayesian framework for inference when the data-generating process may be misspecified, by formulating inference as loss minimization and constructing Gibbs/posterior distributions via Dirichlet-process–based Bayesian non-parametric updating. It presents two computationally efficient, optimization-based updating schemes (prior-to-posterior and predictive-to-posterior) that converge to the same posterior under exchangeability and are shown to be consistent and asymptotically normal when misspecification is present. A calibration procedure for the learning rate ensures valid uncertainty quantification, and the authors apply the methods to causal inference via propensity-score regression, demonstrating doubly robust properties and robust performance in simulations and an empirical UK speed camera study. The results offer scalable, robust alternatives to traditional likelihood-based Bayesian updating in settings where model misspecification is a concern, with broad applicability to causal inference and predictive decision-making.

Abstract

In the usual Bayesian setting, a full probabilistic model is required to link the data and parameters, and the form of this model and the inference and prediction mechanisms are specified via de Finetti's representation. In general, such a formulation is not robust to model misspecification of its component parts. An alternative approach is to draw inference based on loss functions, where the quantity of interest is defined as a minimizer of some expected loss, and to construct posterior distributions based on the loss-based formulation; this strategy underpins the construction of the Gibbs posterior. We develop a Bayesian non-parametric approach; specifically, we generalize the Bayesian bootstrap, and specify a Dirichlet process model for the distribution of the observables. We implement this using direct prior-to-posterior calculations, but also using predictive sampling. We also study the assessment of posterior validity for non-standard Bayesian calculations. We show that the developed non-standard Bayesian updating procedures yield valid posterior distributions in terms of consistency and asymptotic normality under model misspecification. Simulation studies show that the proposed methods can recover the true value of the parameter under misspecification.

Paper Structure

This paper contains 33 sections, 4 theorems, 60 equations, 4 figures, 7 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Validity of Bayesian inference via loss functions
  3. Correct specification and misspecification
  4. Targeting parameters via loss minimization
  5. Assessing the validity of non-standard posterior inference
  6. Posterior calculation via loss functions
  7. Prior-to-posterior updating via Bayesian non-parametric modelling
  8. Predictive-to-posterior updating via Bayesian non-parametric modelling
  9. Verifying the validity of the DP-based loss calculation
  10. Calibrating uncertainty via a valid posterior inference
  11. Asymptotic results
  12. Doubly robust causal inference via PS regression
  13. Frequentist inference in PS regression
  14. Bayesian inference in PS regression
  15. Simulation studies
  16. ...and 18 more sections

Key Result

Theorem 1

Suppose the prior $\pi_0(\theta)$ has full Hellinger support, and $\ell\left(z,\theta\right)$ is continuous $\forall\theta \in \Theta$ with Let $\theta^s_1 = \theta(\varpi^s,\zeta^{s})$ be the unique solution to for any given $\varpi^s$ and $\zeta^s$ generated from Algorithm A0. Let $\theta^s_2=\theta^s(z^s)$ be the unique solution to $\min\limits_{\theta \in \Theta} \sum_{i=1}^{N} \ell\left(z^{

Figures (4)

  • Figure 1: Density plots of $H$ with $n=100$ (left) and $n=10000$ (right). The solid, dashed, dotted and dotted dash lines represent results from $\alpha=0,1,10,100$ respectively.
  • Figure 2: Checking coverage validity using Monahan & Boos: Density plots of $H$ with $n=100$ (left) and $n=10000$ (right) under a mis-specified model. The solid, dashed, dotted and dotted dash lines represent results from $\alpha=0,1,10,100$ respectively.
  • Figure 3: UK speed camera data: Posterior predictive densities using three different methods.
  • Figure : Prior-to-posterior inference based on a stick-breaking process.

Theorems & Definitions (11)

  • Definition 1
  • Definition 2
  • Theorem 1
  • Theorem 2
  • Corollary 1
  • Definition 3
  • proof
  • proof
  • proof
  • Theorem 3
  • ...and 1 more