NAUTILUS: boosting Bayesian importance nested sampling with deep learning

TL;DR

This work tackles the inefficiency of Bayesian posterior and evidence estimation in astronomy by boosting importance nested sampling (INS) with neural networks. The authors introduce NAUTILUS, a dynamic INS framework that uses neural-network bounds and multi-ellipsoid decomposition to produce high-quality proposals and exploit all likelihood evaluations, achieving substantial gains in sampling efficiency and accuracy across synthetic and real-world problems. The approach demonstrates strong performance relative to NS and MCMC baselines, scales reasonably with dimensionality, and benefits from straightforward parallelization, making it a practical tool for expensive likelihoods such as exoplanet RVs and cosmological SED fitting. An open-source implementation and careful discussion of convergence, overhead, and hyperparameter sensitivity support broad applicability in astrophysical Bayesian inference.

Abstract

We introduce a novel approach to boost the efficiency of the importance nested sampling (INS) technique for Bayesian posterior and evidence estimation using deep learning. Unlike rejection-based sampling methods such as vanilla nested sampling (NS) or Markov chain Monte Carlo (MCMC) algorithms, importance sampling techniques can use all likelihood evaluations for posterior and evidence estimation. However, for efficient importance sampling, one needs proposal distributions that closely mimic the posterior distributions. We show how to combine INS with deep learning via neural network regression to accomplish this task. We also introduce NAUTILUS, a reference open-source Python implementation of this technique for Bayesian posterior and evidence estimation. We compare NAUTILUS against popular NS and MCMC packages, including EMCEE, DYNESTY, ULTRANEST and POCOMC, on a variety of challenging synthetic problems and real-world applications in exoplanet detection, galaxy SED fitting and cosmology. In all applications, the sampling efficiency of NAUTILUS is substantially higher than that of all other samplers, often by more than an order of magnitude. Simultaneously, NAUTILUS delivers highly accurate results and needs fewer likelihood evaluations than all other samplers tested. We also show that NAUTILUS has good scaling with the dimensionality of the likelihood and is easily parallelizable to many CPUs.

Figures (10)

• Figure 1: Schematic view of the exploration phase of the INS algorithm. We start by sampling the entire prior uniformly with random points. Then, the points with the highest likelihood are identified, and the volume they represent is sampled further. This step is repeated until a convergence criterion is met.
• Figure 2: Schematic view of how the INS algorithm estimates the posterior. For each shell, the volume is sampled uniformly so we can assign a corresponding sampling density $g$. We then assign an importance weight to each point proportional to the ratio of its likelihood $\mathcal{L}$ to the sampling density $g$. The total posterior is then estimated by summing the contributions from all shells. Although we receive a weighted posterior sample, for visualization purposes, we downsampled points in the above figure according to the importance weight to receive an equal-weighted posterior sample.
• Figure 3: Diagram depicting how new proposal volumes during the exploration phase are constructed. For this example, we chose the two-dimensional Rosenbrock likelihood. The steps are as follows. (1) The set of $N_{\rm live}$ points with the highest likelihood, the so-called live set, is identified. (2) One or multiple non-overlapping bounding ellipsoids are drawn around the live set. (3) The coordinates $\mathbf{\Theta}$ of points in the ellipsoid are transformed into the ellipsoid coordinates $\mathbf{\tilde{\Theta}}$ using a Cholesky decomposition. Similarly, likelihood values are converted into likelihood scores $0 \leq s_{\mathcal{L}} \leq 1$. (4) The transformed coordinates and likelihood scores are used to train a neural network. (5) A cut $\hat{s}_{\mathcal{L}, \rm min}$ in the predicted likelihood score is determined that corresponds to the likelihood score of the live set. (6) The new proposal volume is defined as that part of the bounding ellipsoid where the predicted likelihood score is above $\hat{s}_{\mathcal{L}, \rm min}$.
• Figure 4: Number of likelihood evaluations (left) and sampling efficiency (right) for each sampler on a given problem. The sampling efficiency is defined as the effective sample size $N_{\rm eff}$ divided by the number of likelihood evaluations. All samplers were run in their default configurations.
• Figure 5: Estimated marginalised posterior distribution of the tenth parameter of the $30$-dimensional LogGamma distribution. In all cases, the results of individual samplers are averaged over repeated runs. We also overplot the analytic result. The bottom panel shows the ratio of the estimated marginal posterior distributions to the analytic result.