Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Demonstrating the power and flexibility of variational assumptions for amortized neural posterior estimation in environmental applications

Elliot Maceda, Emily C. Hector, Amanda Lenzi, Brian J. Reich

TL;DR

This paper proposes a framework for Bayesian posterior estimation by mapping data to posteriors of parameters using a neural network trained on data simulated from the complex model, and shows theoretically that its posteriors converge to the true posteriors in Kullback-Leibler divergence.

Abstract

Classic Bayesian methods with complex models are frequently infeasible due to an intractable likelihood. Simulation-based inference methods, such as Approximate Bayesian Computing (ABC), calculate posteriors without accessing a likelihood function by leveraging the fact that data can be quickly simulated from the model, but converge slowly and/or poorly in high-dimensional settings. In this paper, we propose a framework for Bayesian posterior estimation by mapping data to posteriors of parameters using a neural network trained on data simulated from the complex model. Posterior distributions of model parameters are efficiently obtained by feeding observed data into the trained neural network. We show theoretically that our posteriors converge to the true posteriors in Kullback-Leibler divergence. Our approach yields computationally efficient and theoretically justified uncertainty quantification, which is lacking in existing simulation-based neural network approaches. Comprehensive simulation studies highlight our method's robustness and accuracy.

Demonstrating the power and flexibility of variational assumptions for amortized neural posterior estimation in environmental applications

TL;DR

This paper proposes a framework for Bayesian posterior estimation by mapping data to posteriors of parameters using a neural network trained on data simulated from the complex model, and shows theoretically that its posteriors converge to the true posteriors in Kullback-Leibler divergence.

Abstract

Classic Bayesian methods with complex models are frequently infeasible due to an intractable likelihood. Simulation-based inference methods, such as Approximate Bayesian Computing (ABC), calculate posteriors without accessing a likelihood function by leveraging the fact that data can be quickly simulated from the model, but converge slowly and/or poorly in high-dimensional settings. In this paper, we propose a framework for Bayesian posterior estimation by mapping data to posteriors of parameters using a neural network trained on data simulated from the complex model. Posterior distributions of model parameters are efficiently obtained by feeding observed data into the trained neural network. We show theoretically that our posteriors converge to the true posteriors in Kullback-Leibler divergence. Our approach yields computationally efficient and theoretically justified uncertainty quantification, which is lacking in existing simulation-based neural network approaches. Comprehensive simulation studies highlight our method's robustness and accuracy.
Paper Structure (24 sections, 2 theorems, 19 equations, 14 figures, 4 tables, 1 algorithm)

This paper contains 24 sections, 2 theorems, 19 equations, 14 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Classic simulation based inference
  4. Neural-network based point estimation
  5. Neural-network based posterior estimation
  6. Our Contribution
  7. The VaNBayes Approach to Amortized NPE
  8. Model and Estimation
  9. Choosing the proposal distribution
  10. Theoretical Guarantees/Choosing the variational posterior distribution
  11. Evaluating the fit of the posterior approximation
  12. Numerical Illustrations
  13. Multiple Linear Regression
  14. Sparse linear regression
  15. Autologistic regression model
  16. ...and 9 more sections

Key Result

theorem 1

Given that $\Pi( \hbox{\boldmath $\theta$})$ is a valid distribution of $\hbox{\boldmath $\theta$}$ with the same support as the prior for our model, $\pi( \hbox{\boldmath $\theta$})$, the weights $\widehat{ \hbox{\bf W}}$ chosen by VaNBayes are asymptotically (in $N$) invariant to the training dist

Figures (14)

  • Figure 1: The posterior medians of Bayesflow and VaNBayes approximate marginal posteriors of each of the covariates with the true value (red dot) in the $p=25$ case.
  • Figure 2: The mean absolute error averaged across the covariates for VaNBayes and Bayesflow as a function of the number of covariates, $p$. The dashed lines are the 95% confidence intervals.
  • Figure 3: Sampling distribution of the posterior inclusion probabilities ($\hbox{PIP}_j$) from MCMC versus the proposed VaNBayes method over 100 simulated datasets from the sparse linear regression model with $p=10$ (left) and $p=20$ (right) and true model that include only variables 1, 2 and 6.
  • Figure 4: Posterior median of $\sigma$ for the proposed method and MCMC for $p=10$ (left) and $p=20$ (right). Each point is one simulated dataset.
  • Figure 5: QQ-plot of the probability integral transform statistics for the autologistic regression coefficients, $\beta_j$, and log dependence parameter, $\log(\phi)$.
  • ...and 9 more figures

Theorems & Definitions (3)

  • theorem 1
  • theorem 2
  • proof