The Statistical Accuracy of Neural Posterior and Likelihood Estimation

TL;DR

While NPE and NLE methods are just as accurate as ABC and BSL, it is proved that this accuracy can often be achieved at a vastly reduced computational cost, and will therefore deliver more attractive approximations than ABC and BSL in certain problems.

Abstract

Neural posterior estimation (NPE) and neural likelihood estimation (NLE) are machine learning approaches that provide accurate posterior, and likelihood, approximations in complex modeling scenarios, and in situations where conducting amortized inference is a necessity. While such methods have shown significant promise across a range of diverse scientific applications, the statistical accuracy of these methods is so far unexplored. In this manuscript, we give, for the first time, an in-depth exploration on the statistical behavior of NPE and NLE. We prove that these methods have similar theoretical guarantees to common statistical methods like approximate Bayesian computation (ABC) and Bayesian synthetic likelihood (BSL). While NPE and NLE methods are just as accurate as ABC and BSL, we prove that this accuracy can often be achieved at a vastly reduced computational cost, and will therefore deliver more attractive approximations than ABC and BSL in certain problems. We verify our results theoretically and in several examples from the literature.

Figures (9)

• Figure 1: Univariate posterior approximations of the rate parameter $\lambda$ (true value $\lambda = 100$, shown by the dashed vertical line) for a dataset generated from the stereological model with $n = 1000$ observations. The NPE approximations are compared across varying numbers of simulations against the posterior obtained via ABC-SMC.
• Figure 2: Bias of the posterior mean for $\lambda$ visualized through boxplots across varying $n$ and $N$.
• Figure 3: Comparison of $\mathrm{KLD}\{\Pi(\cdot\mid S_n)\|\widehat{Q}_n(\cdot\mid S_n)\}$ across different choices of $N$ for extreme (a) and minor (b) cases of incompatibility. Please note that the scales in Panels (a) and (b) differ markedly.
• Figure 4: Bias of the posterior mean for $g$ visualized through boxplots for $n=1000$ and $n=5000$ across varying $N$.
• Figure 5: Comparison of $\mathrm{KLD}\{\Pi(\cdot\mid S_n)\|\widehat{Q}_n(\cdot\mid S_n)\}$ across different choices of $N$, for summaries using octiles and hexadeciles.

Theorems & Definitions (32)

• Definition 1
• Lemma 1: Pinsker
• Definition 2: NPE
• Remark 1
• Remark 2
• Remark 3
• Remark 4
• Remark 5
• Remark 6
• Theorem 1