Multifidelity Simulation-based Inference for Computationally Expensive Simulators

TL;DR

The paper tackles the challenge of inferring parameters for computationally expensive high-fidelity simulators in SBI. It introduces MF-(TS)NPE, a multifidelity framework that pre-trains neural density estimators on cheap low-fidelity data and refines them with limited high-fidelity simulations, including a sequential TSNPE variant and an active-learning extension A-MF-TSNPE. Across standard SBI benchmarks and neuroscience-intensive tasks, MF-(TS)NPE achieves similar posterior accuracy with orders of magnitude fewer high-fidelity simulations compared to baselines. The work highlights transfer learning and active acquisition as key ingredients for efficient SBI on expensive simulators and discusses limitations, future directions, and the promise of broader adoption.

Abstract

Across many domains of science, stochastic models are an essential tool to understand the mechanisms underlying empirically observed data. Models can be of different levels of detail and accuracy, with models of high-fidelity (i.e., high accuracy) to the phenomena under study being often preferable. However, inferring parameters of high-fidelity models via simulation-based inference is challenging, especially when the simulator is computationally expensive. We introduce MF-(TS)NPE, a multifidelity approach to neural posterior estimation that uses transfer learning to leverage inexpensive low-fidelity simulations to efficiently infer parameters of high-fidelity simulators. MF-(TS)NPE applies the multifidelity scheme to both amortized and non-amortized neural posterior estimation. We further improve simulation efficiency by introducing A-MF-TSNPE, a sequential variant that uses an acquisition function targeting the predictive uncertainty of the density estimator to adaptively select high-fidelity parameters. On established benchmark and neuroscience tasks, our approaches require up to two orders of magnitude fewer high-fidelity simulations than current methods, while showing comparable performance. Overall, our approaches open new opportunities to perform efficient Bayesian inference on computationally expensive simulators.

Figures (19)

• Figure 1: Multifidelity Neural Posterior Estimation proceeds by dense sampling from the prior distribution, running the low-fidelity simulator (e.g., a two-compartment neuron modelhodgkin_quantitative_1952), and training a neural density estimator with a negative log-likelihood loss. MF-NPE then retrains the pre-trained network on sparse samples from the same prior distribution and respective high-fidelity simulations (e.g., a multicompartmental neuron modelrall_theoretical_1995). Given empirical observations $\boldsymbol{x_o}$, MF-NPE estimates the posterior distribution given the high-fidelity model. In the sequential case, the parameters for high-fidelity simulations are drawn from iterative refinements of the prior distribution within the support of the current posterior estimate, at some observation $\boldsymbol{x_o}$.
• Figure 2: (A)-MF-(TS)NPE outperforms NPE and TSNPE in simulation-efficiency. C2ST and MMD averaged over 10 network initializations with means and $95 \%$ confidence intervals. MF-NPE 4 and MF-NPE 5 are pretrained on $10^4$ and $10^5$ low-fidelity simulations, respectively. Results for the GaussianBlob task in Fig. \ref{['fig:gaussian_blobs_eval']}; variations on the OU task and comparisons to MF-ABC in Fig. \ref{['figure:ou_extended_comparison']}.
• Figure 3: MF-NPE improves inference on neuroscience tasks.(A) Thick-tufted layer 5 pyramidal cell from the neocortex. (B) Performance evaluation with NLTP (same naming convention as in Fig. \ref{['figure:task1']}). Amortized methods are averaged over 10 network initializations; non-amortized trained once per 100 observations. Similar results were obtained with NRMSE (Appendix \ref{['app:nrmse-neuro']}). MF-NPE, and especially its sequential variants, are orders of magnitude more simulation-efficient than NPE. (C) Simulation-based calibration for NPE and MF-NPE trained on $10^3$ high-fidelity samples. Posterior samples and predictives are in Appendix \ref{['appendix:multicompartmental']}. (D) Schematic of the low and high-fidelity models of a spiking network. (E) Performance of NPE and MF-NPE evaluated on 10000 true observations with NLTP: averages over 10 network initializations, and 95% confidence intervals. (F) Proportion of posterior samples within the target firing rate bounds. MF-NPE produces a higher fraction of parameter sets within the bounds than NPE.
• Figure 4: (A) Schematic figure representing lower bound on transfer error ($1 / \text{MF-NPE{} performance}$) as a function of mutual information between the low- and high-fidelity models, given a fixed simulation budget. (B) Uncertainty coefficient monotonically decreases with noise parameter $\delta$ and is invariant to data inversion. (C) Empirical results with MF-NPE support the hypothesis that transfer performance is dependent on both mutual information and representational coherence. Note that NPE---with the same high-fidelity simulation budget of $10^2$---has similar performance as MF-NPE in the case where the low- and high-fidelity models have low mutual information.
• Figure 5: The four parameters of the Ornstein-Uhlenbeck process: the mean $\mu$, standard deviation $\sigma$, convergence rate $\gamma$, and $\mu_\mathrm{offset}$, which is the difference between the initial condition $X(0)$ and mean $\mu$.