Multilevel neural simulation-based inference

TL;DR

The paper tackles Bayesian inference with simulator-based models where the likelihood is intractable and simulations are expensive. It introduces a multilevel Monte Carlo (MLMC) framework to combine hierarchies of simulators with costs $C_0< C_1< \dots < C_L$, forming a telescoping objective that enables efficient training of neural likelihood (NLE) and neural posterior (NPE) estimators. The authors prove variance bounds for MLMC terms and provide guidance for optimal sample allocation across fidelity levels, along with gradient-stability techniques for training. Empirically, ML-NLE and ML-NPE outperform conventional MC-based SBI across g-and-k, Ornstein–Uhlenbeck, toggle-switch, and CAMELS cosmology tasks, achieving similar or better accuracy with substantially fewer high-fidelity simulations. The work demonstrates that MLMC-based SBI is broadly applicable, complementary to other compute-efficient SBI methods, and capable of handling more than two fidelity levels, with practical impact for expensive simulations in physics, biology, and cosmology.

Abstract

Neural simulation-based inference (SBI) is a popular set of methods for Bayesian inference when models are only available in the form of a simulator. These methods are widely used in the sciences and engineering, where writing down a likelihood can be significantly more challenging than constructing a simulator. However, the performance of neural SBI can suffer when simulators are computationally expensive, thereby limiting the number of simulations that can be performed. In this paper, we propose a novel approach to neural SBI which leverages multilevel Monte Carlo techniques for settings where several simulators of varying cost and fidelity are available. We demonstrate through both theoretical analysis and extensive experiments that our method can significantly enhance the accuracy of SBI methods given a fixed computational budget.

Figures (12)

• Figure 1: Low- and high-fidelity cosmological simulations from the CAMELS data CAMELS_DR1 studied in \ref{['sec:cosmology']}.
• Figure 2: Performance of our ML-NLE and ML-NPE method on the g-and-k example. (a) KL-divergence ($\downarrow$) between the estimated and the (almost) exact density for ML-NLE under different high-fidelity samples $n_1$. We compare it with NLE (low) trained on only low-fidelity data ($n=10^4$) and NLE (high), trained on only high-fidelity data ($n=300$). (b) Negative log-posterior density (NLPD $\downarrow$) for ML-NPE, NPE (low) with $n=10^3$, and NPE (high) with $n=100$. (c) One instance of learned densities using NLE. (d) Empirical coverage plot for ML-NPE, NPE (low), and NPE (high).
• Figure 3: NLPD and KLD for ML-NPE (ours) and TL-NPEKrouglova2025 using different patience values (hyperparameter of TL-NPE) over $20$ runs. ML-NPE performs better in the limited high-fidelity data scenario ($n_1 = 10$) (a), whilst being competitive when $n_1 = 100$ (b). The choice of hyperparameter in TL-NPE depends upon the number of high-fidelity samples available.
• Figure 4: MMD ($\downarrow$) across 5000 parameter values for NLE and ML-NLE.
• Figure 5: NLPD $\downarrow$ and empirical coverage of NPE and ML-NPE for the cosmological inference task.

Theorems & Definitions (6)

• Theorem 1
• Theorem 2
• proof
• proof
• Theorem 3
• proof