Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Asymptotics of cut distributions and robust modular inference using Posterior Bootstrap

Emilia Pompe, Mikołaj J. Kasprzak, Pierre E. Jacob

TL;DR

An algorithm based on the Posterior Bootstrap that delivers credible regions with the nominal frequentist asymptotic coverage is proposed and a Bernstein-von Mises theorem is obtained as well as a Laplace approximation with quantitative bounds are obtained.

Abstract

Bayesian inference provides a framework to combine various model components with shared parameters, allowing joint uncertainty estimation and the use of all available data sources. Unfortunately, misspecification of any part of the model might propagate to all other parts and can lead to unsatisfactory results. Cut distributions have been proposed as a remedy, where the information is prevented from flowing along certain directions. We study cut distributions from an asymptotic perspective and obtain a Bernstein-von Mises theorem, as well as a Laplace approximation with quantitative bounds. We then propose an algorithm based on the Posterior Bootstrap that delivers credible regions with the nominal frequentist asymptotic coverage. The proposed methods are illustrated with numerical experiments in a variety of examples, including causal inference with propensity scores.

Asymptotics of cut distributions and robust modular inference using Posterior Bootstrap

TL;DR

An algorithm based on the Posterior Bootstrap that delivers credible regions with the nominal frequentist asymptotic coverage is proposed and a Bernstein-von Mises theorem is obtained as well as a Laplace approximation with quantitative bounds are obtained.

Abstract

Bayesian inference provides a framework to combine various model components with shared parameters, allowing joint uncertainty estimation and the use of all available data sources. Unfortunately, misspecification of any part of the model might propagate to all other parts and can lead to unsatisfactory results. Cut distributions have been proposed as a remedy, where the information is prevented from flowing along certain directions. We study cut distributions from an asymptotic perspective and obtain a Bernstein-von Mises theorem, as well as a Laplace approximation with quantitative bounds. We then propose an algorithm based on the Posterior Bootstrap that delivers credible regions with the nominal frequentist asymptotic coverage. The proposed methods are illustrated with numerical experiments in a variety of examples, including causal inference with propensity scores.

Paper Structure

This paper contains 101 sections, 14 theorems, 321 equations, 5 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Cutting feedback in Bayesian models
  3. Main questions
  4. Contributions
  5. Asymptotics of modular inference
  6. Setting and notation
  7. Asymptotics of two-step M-estimators
  8. Bernstein-von Mises for the cut posterior
  9. Independent data sets of different sizes
  10. Laplace approximation and its error
  11. Construction of the Laplace approximation
  12. Error of the Laplace approximation
  13. Explicit error bound in a biased data setting
  14. Posterior Bootstrap for Modular inference
  15. Algorithm
  16. ...and 86 more sections

Key Result

Theorem 1

Let $(\hat{\theta}_1,\hat{\theta}_2)$ be the 2SM-estimator in eq:2sM and suppose that $\hat{\theta}\xrightarrow{n\to\infty}\theta^*$ in $P^*$-probability, where $\theta^*$ is defined in eq:pseudotrue and lies in the interior of $\Theta_1\times\Theta_2$. Assume assump:priors_asymptotic-assump:integra where $J_1^*, J_2^*, R_J^*$ are given by eq:j_defs. Then, denoting by $\tilde{\pi}_{\rm cut}$ the l

Figures (5)

  • Figure 1: Graphical representation of the model \ref{['eq:general_model']}, made of two modules. The parameters are $\theta_1$ and $\theta_2$, and the data sets are $x_1$ and $x_2$.
  • Figure 2: Standard Posterior distribution, its Laplace approximation, cut posterior distribution, and its Laplace approximation, and PBMI for model \ref{['model:toy_example_v2']} under different scenarios.
  • Figure 3: Histograms of income of the participants of the study, pre-intervention (left panel) and post-intervention (right panel) for the treated and the non-treated group. The histograms are normalized by group. Vertical segments represent the median of each group.
  • Figure 4: Left: propensity scores $(X_i^T \theta_1)$ where $\theta_1$ is set to the MAP in the first module. Right: density plot of $\beta_{Z}$, the regression coefficient associated with the effect of treatment, drawn from the cut posterior and PBMI with and without weight refreshment, for model \ref{['model:linear_causal_v2']}.
  • Figure 5: Standard Posterior distribution, cut posterior, Cut-Laplace and PBMI, for $\theta_{2,1}$ (left) and $\theta_{2,2}$ (right) in the epidemiological example of \ref{['section:epidemiological_study_v2']}.

Theorems & Definitions (31)

  • Theorem 1
  • Example 1
  • Theorem 2
  • Remark 1
  • Remark 2
  • Remark 3
  • Example 2
  • Proposition 1
  • Remark 4
  • Theorem 3
  • ...and 21 more