Importance Nested Sampling and the MultiNest Algorithm

TL;DR

The paper tackles the computational burden of Bayesian evidence calculation and multimodal posteriors by enhancing MultiNest with Importance Nested Sampling (INS). INS reuses all samples via a pseudo-importance density to achieve substantially higher evidence accuracy without changing the underlying sampling, enabling reliable model selection and parameter inference in difficult problems. Through tests on Gaussian shells, egg-box, and high-dimensional Gaussian mixtures, INS consistently improves log-evidence accuracy over vanilla NS, though it increases memory requirements and can show biases in certain constant-efficiency regimes. The work provides practical guidance on configuring live points and efficiency and demonstrates that INS broadens the practical applicability of Nested Sampling in cosmology and astrophysical data analysis.

Abstract

Bayesian inference involves two main computational challenges. First, in estimating the parameters of some model for the data, the posterior distribution may well be highly multi-modal: a regime in which the convergence to stationarity of traditional Markov Chain Monte Carlo (MCMC) techniques becomes incredibly slow. Second, in selecting between a set of competing models the necessary estimation of the Bayesian evidence for each is, by definition, a (possibly high-dimensional) integration over the entire parameter space; again this can be a daunting computational task, although new Monte Carlo (MC) integration algorithms offer solutions of ever increasing efficiency. Nested sampling (NS) is one such contemporary MC strategy targeted at calculation of the Bayesian evidence, but which also enables posterior inference as a by-product, thereby allowing simultaneous parameter estimation and model selection. The widely-used MultiNest algorithm presents a particularly efficient implementation of the NS technique for multi-modal posteriors. In this paper we discuss importance nested sampling (INS), an alternative summation of the MultiNest draws, which can calculate the Bayesian evidence at up to an order of magnitude higher accuracy than `vanilla' NS with no change in the way MultiNest explores the parameter space. This is accomplished by treating as a (pseudo-)importance sample the totality of points collected by MultiNest, including those previously discarded under the constrained likelihood sampling of the NS algorithm. We apply this technique to several challenging test problems and compare the accuracy of Bayesian evidences obtained with INS against those from vanilla NS.

Figures (6)

• Figure 1:
• Figure 2: Illustrations of the ellipsoidal decompositions returned by MultiNest. The points given as input are overlaid on the resulting ellipsoids. Here 1000 points were sampled uniformly from: (a) two non-intersecting ellipsoids; and (b) a torus.
• Figure 3: Test problem 1: (a) two-dimensional plot of the likelihood function defined in Eqs. (\ref{['eq:gshellL']}) and (\ref{['eq:gshell']}); (b) dots denoting the points with the lowest likelihood at successive iterations of the MultiNest algorithm.
• Figure 4:
• Figure 5: Test problem 3: Marginalized posterior distribution in the first 2 dimensions of the 16D Gaussian mixture model discussed in Sec. \ref{['sec:applications:gmix']}. Panel (a) shows the analytical distribution while panel (b) shows the distribution obtained from MultiNest. The contours represent the 68% and 95% Bayesian credible regions.