Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards a Field Based Bayesian Evidence Inference from Nested Sampling Data

Margret Westerkamp, Jakob Roth, Philipp Frank, Will Handley, Torsten Enßlin

TL;DR

This work proposes to transform the prior volume estimation into a Bayesian inference problem, which allows it to incorporate a smoothness assumption for likelihood–prior–volume relations and aims to increase the accuracy of the volume estimates and thus improve the overall log-evidence calculation using NS.

Abstract

Nested sampling (NS) is a stochastic method for computing the log-evidence of a Bayesian problem. It relies on stochastic estimates of prior volumes enclosed by likelihood contours, which limits the accuracy of the log-evidence calculation. We propose to transform the prior volume estimation into a Bayesian inference problem, which allows us to incorporate a smoothness assumption for likelihood-prior volume relations. As a result, we aim to increase the accuracy of the volume estimates and thus improve the overall log-evidence calculation using NS. The method presented works as a post-processing step for NS and provides posterior samples of the likelihood-prior-volume relation, from which the log-evidence can be calculated. We demonstrate an implementation of the algorithm and compare its results with plain NS on two synthetic datasets for which the underlying evidence is known. We find a significant improvement in accuracy for runs with less than one hundred active samples in NS, but are prone to numerical problems beyond this point.

Towards a Field Based Bayesian Evidence Inference from Nested Sampling Data

TL;DR

This work proposes to transform the prior volume estimation into a Bayesian inference problem, which allows it to incorporate a smoothness assumption for likelihood–prior–volume relations and aims to increase the accuracy of the volume estimates and thus improve the overall log-evidence calculation using NS.

Abstract

Nested sampling (NS) is a stochastic method for computing the log-evidence of a Bayesian problem. It relies on stochastic estimates of prior volumes enclosed by likelihood contours, which limits the accuracy of the log-evidence calculation. We propose to transform the prior volume estimation into a Bayesian inference problem, which allows us to incorporate a smoothness assumption for likelihood-prior volume relations. As a result, we aim to increase the accuracy of the volume estimates and thus improve the overall log-evidence calculation using NS. The method presented works as a post-processing step for NS and provides posterior samples of the likelihood-prior-volume relation, from which the log-evidence can be calculated. We demonstrate an implementation of the algorithm and compare its results with plain NS on two synthetic datasets for which the underlying evidence is known. We find a significant improvement in accuracy for runs with less than one hundred active samples in NS, but are prone to numerical problems beyond this point.
Paper Structure (12 sections, 42 equations, 5 figures, 4 tables)

This paper contains 12 sections, 42 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Nested Sampling Algorithm
  4. Information Field Theory
  5. Bayesian Inference of the likelihood-prior-volume Function
  6. Results and Analysis
  7. Gaussian Case
  8. Spike & Slab Case
  9. Discussion
  10. Reparametrisation
  11. Maximal Likelihood Calculation
  12. Inference with Gaussian Approximation

Figures (5)

  • Figure S1: Illustration of NS output for simple Gaussian example introduced in skilling_nested_2006 and further elaborated in section \ref{['sec:Gaussian']} with two live points. The left side shows the full NS data generated with anesthetic and the right side shows a zoomed in section, which is indicated on the left. The zoomed in image additionally shows the information of the likelihood dead contours $\vec{d}_L$, which we use as data for the Bayesian inference of the prior volumes. In both figures, the samples of likelihood-prior-volume functions defined by prior volume samples, $X_k$, $\vec{d}_L(X_k)$, are shown as well as a likelihood-prior-volume function defined by the deterministic NS approach in eq. \ref{['eq:MeanPriorVolume']}, $\vec{d}_L(\bar{X})$.
  • Figure : (a) Change in $a$:
  • Figure S3: Illustration of the Bayesian field inference process for the simple Gaussian, which is further elaborated in section \ref{['sec:Gaussian']} for two live points. The prior samples, the data used and the final reconstruction compared to the NS approach and the ground truth are shown. Figure \ref{['fig:IFTPrior']}: Prior samples for the likelihood-prior-volume function ($L^*(X)$) in yellow together with the ground truth. Besides a zoom area is marked (the same area as in figure \ref{['fig:GaussexampleFull']}) which is taken to zoom into the data in figure \ref{['fig:IFTLikelihood']} and the reconstruction in figure \ref{['fig:IFTPosteriorZoom']}. Figure \ref{['fig:IFTLikelihood']}: Data on likelihood dead contours for the given zoom area. Figure \ref{['fig:IFTPosteriorZoom']} and figure \ref{['fig:IFTPosterior']}: Reconstruction mean (rec mean) of the likelihood-prior-volume function and the associated uncertainty, defined via the onesigma contours (rec uncertainty), zoomed in \ref{['fig:IFTPosteriorZoom']} and full image in \ref{['fig:IFTPosterior']}. Moreover, the result for the likelihood-prior-volume function for the deterministic NS approach is shown ($d_{L}(\bar{X})$), which is the same as in figure \ref{['fig:GaussexampleZoom']}, and the analytic likelihood-prior-volume-function (ground truth).
  • Figure S4: Reconstruction results for the Gaussian prior volumes accompanied by the likelihood contours given by NS on the left and the computed log-evidence on the right for $n_\text{live} \in \{2, 10, 1000\}$ from top to bottom. The inferred posterior samples (rec samples) are shown together with their mean (rec mean) and compared with the corresponding statistical (NS samples) and deterministic (det NS mean) NS results and the ground truth.
  • Figure S5: Reconstruction results for the spike-and-slab prior volumes accompanied by the likelihood contours given by NS on the left and the computed log-evidence on the right for $n_\text{live} \in \{2, 10, 1000\}$ from top to bottom. The inferred posterior samples (rec samples) are shown together with their mean (rec mean) and compared with the corresponding statistical (NS samples) and deterministic (det NS mean) NS results and the ground truth.